\documentclass[twoside,11pt]{article} % Package standard : Utilisation de caracteres accentues, mode francais et graphique \usepackage{url} \usepackage[latin1]{inputenc} \usepackage[T1]{fontenc} \usepackage[english]{babel} \usepackage{graphicx} % package a mettre pour faire du pdf \usepackage{palatino} % Extension de symboles mathematiques \usepackage{amssymb} % Definition pour Docs Sophya \usepackage{defsophya} \begin{document} \begin{titlepage} % The title page - top of the page with the title of the paper \titrehp{Sophya \\ An overview } % Authors list \auteurs{ R. Ansari & ansari@lal.in2p3.fr \\ E. Aubourg & aubourg@hep.saclay.cea.fr \\ G. Le Meur & lemeur@lal.in2p3.fr \\ C. Magneville & cmv@hep.saclay.cea.fr \\ S. Henrot-Versille & versille@in2p3.fr } % \auteursall % The title page - bottom of the page with the paper number \titrebp{1} \end{titlepage} \tableofcontents \newpage \section{Introduction} {\bf SOPHYA} ({\bf SO}ftware for {\bf PHY}sics {\bf A}nalysis) is a collection of C++ classes designed for numerical and physics analysis software development. Our goal is to provide easy to use, yet powerful classes which can be used by scientists. We have decided to use as much as possible available numerical analysis libraries, encapsulating them whenever possible. The SOPHYA design and implementation has been carried out with the specific goal of providing the general framework for the Planck-HFI data processing software. However, most of the packages presented here have a more general scope than the CMB analysis and Planck mission problem. The source directory tree \footnote{ CVS server: cvsserver.lal.in2p3.fr:/projects/Eros/CVSPlanck} is organised into a number of modules. \begin{itemize} \item[] {\bf Mgr/} Scripts for code management, makefile generation and software installation \item[] {\bf SysTools/} General architecture support classes such as {\tt PPersist, NDataBlock}, and few utility classes ({\tt DataCard, DVList} \ldots). \item[] {\bf TArray/} template numerical arrays, vectors and matrices \\ ({\tt PixelMap SphericalMap} \ldots) \item[] {\bf NTools/} Some standard numerical analysis tools (linear, and non linear parameter fitting, FFT, \ldots) \item[] {\bf HiStats/} Histogram-ming and data set handling classes \\ ({\tt Histo Histo2D NTuple XNTuple} \ldots) \end{itemize} The modules listed below are more tightly related to the CMB (Cosmic Microwave Background) data analysis problem: \begin{itemize} \item[] {\bf SkyMap/} Local and full sky maps, and few geometry handling utility classes. \\ ({\tt PixelMap, LocalMap, SphericalMap, \ldots}) \item[] {\bf SkyT/} classes for spectral emission and detector frequency response modelling \\ ({\tt SpectralResponse, RadSpectra, BlackBody} \ldots) \item[] {\bf Samba/} Spherical harmonic analysis. \end{itemize} The following modules contain the interface classes with external libraries: \begin{itemize} \item[] {\bf FitsIOServer/} Classes for handling file input-output in FITS format using the cfitsio library. \item[] {\bf LinAlg/} Interface with Lapack linear algebra package \item[] {\bf IFFTW/} Interface with FFTW package (libfftw.a) \end{itemize} Other modules: \begin{itemize} \item[] {\bf Tests/} Simple test programs \item[] {\bf PrgUtil/} Various utility programs (runcxx, scanppf, scanfits, \ldots) \item[] {\bf PMixer/} skymixer and related programs \item[] {\bf ProgPI/} interactive analysis tool - It should be noted that this module uses the SOPHYA class library and is based on {\bf PI} which is a C++ library defining a complete GUI program architecture. An additional module (PIext) define the interactive analysis program framework and the interfaces with the objects in SOPHYA. The {\bf PI/} \footnote{the PI package documentation is available from {\bf http://www.lal.in2p3.fr/recherche/eros/PeidaDoc/} } and {\bf PIext/} modules are not currently part of the SOPHYA CVS structure. \end{itemize} \newpage \section{Using Sophya} Two environment variables {\bf DPCBASEREP} and {\bf EROSCXX} are used to define the path where the Sophya libraries and executable are installed. {\bf DPCBASEREP} defines the base directory path and {\bf EROSCXX} the name of the C++ compiler. The complete path is built using {\bf DPCBASEREP}, the operating system name (as obtained by the {\tt uname} command), and the compiler name. In the example below, we show the complete path for a {\tt Linux} system, using the GNU g++ compiler: \begin{itemize} \item \$DPCBASEREP/Include : Include (.h) files \item \$DPCBASEREP/Linux-g++/Libs : Path for the archive libraries (.a) \item \$DPCBASEREP/Linux-g++/ShLibs : Shared library path (.so) \item \$DPCBASEREP/Linux-g++/Exec : Executable file path \end{itemize} In order to use the shared libraries, the {\bf LD\_LIBRARY\_PATH} variable should contain the Sophya shared library path ({\tt \$DPCBASEREP/Linux-g++/ShLibs } when using g++ compiler on Linux) For modules using external libraries, the {\bf EXTLIBDIR} environment variable should contain the path to these libraries and corresponding include files. C-FitsIO anf FFTW include files should be accessible through: \\ {\tt \$EXTLIBDIR/Include/FitsIO } \\ {\tt \$EXTLIBDIR/Include/FFTW } \\ The corresponding libraries are expected to be found in: \\ {\tt \$EXTLIBDIR/Linux-g++/Libs} \\ The file {\tt \$DPCBASEREP/Include/MakefileUser.h} defines the compilation flags and the list of Sophya libraries. It should be included in the user's makefile. The default compilation rules assumes that the object (.o) and executable files would be put in the following diretories: \\ {\tt \$HOME/`uname`-\$EROSCXX/Objs} \\ {\tt \$HOME/`uname`-\$EROSCXX/Exec}. In the case of a {\tt Linux} system and using {\tt g++} as the C++ compiler, these two directories would be translated to \\ {\tt \$HOME/Linux-g++/Objs} and {\tt \$HOME/Linux-g++/Exec}. The GNU make program should be used. \par The file {\tt \$DPCBASEREP/Include/makefile\_auto} defines the rules to compile a given source program, and link it against the Sophya libraries to produce an executable. The example below shows the steps to compile a program named {\tt trivial.cc }. \begin{verbatim} csh> cp \$DPCBASEREP/Include/makefile_auto makefile csh> make trivial \end{verbatim} This command should compile the {\tt trivial.cc} file, and link it against the sophya libraries. The object and executable file names are: \\ {\tt \$HOME/`uname`-\$EROSCXX/Objs/trivial.o} \\ {\tt \$HOME/`uname`-\$EROSCXX/Exec/trivial}. \par The file {\tt \$DPCBASEREP/Include/makefile\_example} provides another example makefile. Basic usage of Sophya classes are described in in the following sections. Complete Sophya documentation can be found at our web site: \\ {\bf http://hfi-l2.in2p3.fr}. \section{Module SysTools} {\bf SysTools} contains utility classes such as {\tt DataCards} or {\tt DVlist}, an hierarchy of exception classes for Sophya, a template class {\tcls{NDataBlock}} for handling reference counting on numerical arrays, as well as classes providing the services for implementing simple serialization. \vspace*{5mm} \subsection{SOPHYA persistence} \begin{figure}[hbt] \dclsa{PPersist} \dclsbb{PIOPersist}{PInPersist} \dclsb{POutPersist} \caption{partial class diagram for classes handling persistence in Sophya} \end{figure} A simple persistence mechanism is defined in SOPHYA. Its main features are: \begin{itemize} \item[] Portable file format, containing the description of the data structures and object hierarchy. \\ {\bf PPF} {\bf P}ortable {\bf P}ersistence file {\bf F}ormat. \item[] Handling of read/write for mutiply referenced objects. \item[] All write operations are carried using sequential access only. This holds also for read operations, unless positional tags are used. SOPHYA persistence services can thus be used to transfer objects through network links. \end{itemize} The example below shows writing of objects through the use of overloaded operator $ << $ : \begin{verbatim} #include "fiondblock.h" // ... POutPersist pos("aa.ppf"); NDataBlock rdb(40); rdb = 567.89; pos << rdb; // We can also use the PutObject method NDataBlock idb(20); idb = 123; pos.PutObject(idb); \end{verbatim} The following sample programs show the reading of the created PPF file : \begin{verbatim} PInPersist pis("aa.ppf"); NDataBlock rdb; pis >> rdb; cout << rdb; NDataBlock idb; cout << idb; \end{verbatim} \subsection{Using DataCards} The {\bf DataCards} class can be used to read parameters from a file. Each line in the file starting with \@ defines a set of values associated with a keyword. In the example below, we read the parameters corresponding with the keyword {\tt SIZE} from the file {\tt ex.d}. We suppose that {\tt ex.d} contains the line: \\ {\tt @SIZE 400 250} \\ \begin{verbatim} #include "datacards.h" // ... // Initializing DataCards object dc from file ex.d DataCards dc( "ex.d" ); // Getting the first and second parameters for keyword size // We define a default value 100 int size_x = dc.IParam("SIZE", 0, 100); int size_y = dc.IParam("SIZE", 1, 100); cout << " size_x= " << size_x << " size_y= " << size_y << endl; \end{verbatim} \subsection{Dynamic linker} The class {\bf PDynLinkMgr} can be used for managing shared libraries at run time. The example below shows the run time linking of a function:\\ {\tt extern "C" { void myfunc(); } } \\ \begin{verbatim} #include "pdlmgr.h" // ... string soname = "mylib.so"; string funcname = "myfunc"; PDynLinkMgr dyl(soname); DlFunction f = dyl.GetFunction(funcname); if (f != NULL) { // Calling the function f(); } \end{verbatim} \section{Module TArray} {\bf TArray} module contains template classes for handling standard operations on numerical arrays. Using the class {\tt \tcls{TArray} }, it is possible to create and manipulate up to 5-dimension numerical arrays {\tt (int, float, double, complex, \ldots)}. \begin{figure}[hbt] \dclsccc{AnyDataObj}{BaseArray}{\tcls{TArray}} \ldots \\ \dclsccc{\tcls{TArray}}{\tcls{TMatrix}}{\tcls{TVector}} \caption{partial class diagram for arrays, matrices and vectors} \end{figure} \subsection{Using arrays} The example below shows basic usage of arrays: \begin{verbatim} #include "array.h" // ... // Creation , filling of a Matrix TMatrix ma(7,9); ma = RegularSequence(0.1, 0.05); cout << "\n ma = " << ma << endl; \end{verbatim} Example of a simple low-pass filter on a one dimensional array (Vector) \begin{verbatim} // Input Vector containing a noisy periodic signal Vector in(1024), out(1024); in = RandomSequence(RandomSequence::Gaussian, 0., 1.); for(int kk=0; kkPrint(80); Histo *hbx2 = HBandX(1); // Get the second X band (35.Print(80); \end{verbatim} \subsection{Profile Histograms} Profiles histograms contains the mean and the sigma of the distribution of the values filled in each bin. The sigma can be changed to the error on the mean. When filled, the profile histogram looks like a 1D histogram and much of the operations that can be done on 1D histo may be applied onto profile histograms. \subsection{Ntuples} NTuple are memory resident tables of 32 bits floating values (float). They are arranged in columns. Each line is often called an event. These objects are frequently used to analyze data. Graphicals tools (spiapp) can plot a column against an other one with respect to various selection cuts. \\ Here is an example of creation and filling~: \begin{verbatim} #include "ntuple.h" #include "srandgen.h" // ... char* nament[4] = {"i","x","y","ey"}; r_4 xnt[4]; NTuple NT(4,nament); for(i=0;i<5000;i++) { xnt[0] = i+1; xnt[1] = 5.*drandpm1(); // a random value between -5 and +5 xnt[2] = 100.*exp(-0.5*xnt[1]*xnt[1]) + 1.; xnt[3] = sqrt(xnt[2]); xnt[2] += xnt[3] * NorRand(); // add a random gaussian error NT.Fill(xnt); } \end{verbatim} XNtuple are sophisticated NTuple : they accept various types of column values (float,double,int,...) and can be as big as needed (they used buffers on hard disk). \subsection{Writing, seeing \dots } All these objects have been design to be written to or read from a persistant file. The following example shows how to write the previously created objects into such a file~: \begin{verbatim} //-- Writing { char *fileout = "myfile.ppf"; string tag; POutPersist outppf(fileout); tag = "H"; outppf.PutObject(H,tag); tag = "H2"; outppf.PutObject(H2,tag); tag = "NT"; outppf.PutObject(NT,tag); } // closing ``}'' destroy ``outppf'' and automatically close the file ! \end{verbatim} Sophya graphical tools (spiapp) can automatically display and operate all these objects. \section{Module SkyMap} \section{Module NTools} This module provides elementary numerical tools for numerical integration, fitting, sorting and ODE solving. FFTs are also provided (Mayer,FFTPack). \subsection{Fitting} Fitting is done with two classes {\tt GeneralFit} and {\tt GeneralFitData} and is based on the Levenberg-Marquardt method. GeneralFitData is a class which provide a description of the data to be fitted. GeneralFit is the fitter class. Parametrized functions can be given as classes which inherit {\tt GeneralFunction} or as simple C functions. Classes of pre-defined functions are provided (see files fct1dfit.h and fct2dfit.h). The user interface is very close from that of the CERN {\tt Minuit} fitter. Number of objects (Histo, HProf \dots ) are interfaced with GeneralFit and can be easily fitted. \\ Here is a very simple example for fitting the previously created NTuple with a gaussian~: \begin{verbatim} #include "fct1dfit.h" // ... // Read from ppf file NTuple nt; { PInPersist pis("myfile.ppf"); string tag = "NT"; pis.GetObject(nt,tag); } // Fill GeneralData GeneralData mGdata(nt.NEntry()); for(int i=0; i0) {) cout<<"Reduce_Chisquare = "< x(100); TVector y(100); TVector ey(100); for(int i=0;i<100;i++) { x(i) = i; ey(i) = 10.; y(i) = pol((double) i) + ey(i)*NorRand(); ey(i) *= ey(i) } TVector errcoef; Poly polfit; polfit.Fit(x,y,ey,2,errcoef); cout<<"Fit Result"< setenv EROSCXX g++ # Setup the build directory csh> mkdir /usr/local/Sophya/ csh> setenv DPCDEVREP /usr/local/Sophya/ csh> setenv EXTLIBDIR /usr/local/ExtLibs/ # Use the top level makefile in Mgr/ csh> cd \$SRC csh> cp Mgr/Makefile Makefile # Step 1: Create the directory tree and copy the include files (.h) csh> make depend # Step 2: Compile the modules without external library reference csh> make libs # Step 3: Compile the modules WITH external library reference (optional) csh> make extlibs # Step 4: Build libsophya.so csh> make slb # Step 5: Build libextsophya.so (optional) csh> make slbext # Step 6: Compile the PI and PIext modules (optional) csh> make PI # Step 7: Build the corresponding shared library libPI.so (optional) csh> make slbpi \end{verbatim} To compile all modules and build the shared libraries, it is possible to use: \begin{verbatim} # Step 2,3,6 csh> make all # Step 4,5,7 csh> make slball \end{verbatim} At this step, all libraries sould have been made. Programs using Sophya libraries can now be built: \begin{verbatim} # To compile test programs csh> cd Tests csh> make arrt ... csh> cd .. # To compile other programs, for example from the PMixer module csh> cd PMixer csh> make csh> cd .. # To build (s)piapp (libPI.so is needed) csh> cd ProgPI csh> make csh> cd .. \end{verbatim} \subsection{Mgr module} This module contains scripts which can be used for generating the makefiles for each module. \begin{itemize} \item {\bf Makefile} Top level Makefile for builiding the libraries. \item {\bf Makefile.h} contains the defintion of compilation flags for the different compilers and systems. This file is used for building the library and generating {\bf MakefileUser.h} (to be included in makefiles). \item {\bf Makefile.slb} contains the rules for building shared libraries for the different compilers and systems. (to be included in makefiles) \item {\bf crerep\_sophya} c-shell script for creating the directory tree under {\tt \$DPCBASEREP} and {\tt \$DPCDEVREP} \item {\bf install\_sophya} c-shell script for installing the Sophya package. Usually from {\tt \$DPCDEVREP} to {\tt \$DPCBASEREP} \item {\bf mkmflien} c-shell script for making symbolic links or copying include files to {\tt \$DPCDEVREP/Include} or {\tt \$DPCBASEREP/Include} \item {\bf mkmf} c-shell script for generating module makefiles and the top level makefile (named GNUmakefile) \item {\bf mkmflib} c-shell script for generating each library module makefile (named GNUmakefile) \item {\bf mkmfprog} c-shell script for generating makefile for a module containing the source for executable programs (named GNUmakefile). \item {\bf mkmfPI} c-shell script for generating makefile for PI and PIext modules (named GNUmakefile) \item {\bf libdirs} List of Sophya modules without reference to external libraries. \item {\bf extlibdirs} List of Sophya modules with reference to external libraries. \end{itemize} \newpage \appendix \section{Exception handling: An example} For simple programs, it is a good practice to handle the exceptions at least at high level, in the {\tt main()} function. The example below shows the exception handling and the usage of Sophya persistence. \input{ex1.inc} \end{document}