$Id: README,v 1.2 2007/11/16 14:04:05 kmura Exp $
==========================================================================
Geant4 MPI Interface

  Author:
  Koichi Murakami (KEK) / Koichi.Murakami@kek.jp
==========================================================================

About the interface
===================
G4MPI is a native interface with MPI libraries. The directory contains
a Geant4 UI library and a couple of parallelized examples.
Using this interface, users applications can be parllelized with
different MPI compliant libraries, such as LAM/MPI, MPICH2, OpenMPI,
and so on.

System Requirements:
-------------------
* Platform

- Linux (32bit/64bit)
- MacOS (Tiger)


* MPI implementation

- LAM/MPI (7.1.3)
  http://www.lam-mpi.org/

- MPICH2 (1.0.6p1)
  http://www.mcs.anl.gov/research/projects/mpich2/

- Open MPI (1.2.4)
  http://www.open-mpi.org/


* Optional
- ROOT for histogramming/analysis


G4MPI UI library
----------------
G4MPI UI library is provided in the "mpi_interface" directory.
Start with building the G4MPI UI library according to the instruction 
in the directory.


How to make parallel applications:
----------------------------------
* An example of a main program:

#include "G4MPImanager.hh"
#include "G4MPIsession.hh"

int main(int argc,char** argv)
{
  // At first, G4MPImanager/G4MPIsession should be created.
  G4MPImanager* g4MPI= new G4MPImanager(argc,argv);

  // MPI session (G4MPIsession) instead of G4UIterminal
  G4MPIsession* session= g4MPI-> GetMPIsession();

  // LAM users can use G4tcsh.
  G4String prompt= "G4MPI";
  G4UItcsh* tcsh= new G4UItcsh(prompt);
  session-> SetShell(tcsh);

  // user application setting
  G4RunManager* runManager= new G4RunManager();

  // ....

  // After user application setting, just start a MPI session.
  // MPIsession treats both interactive and batch modes.
  session-> SessionStart();

  // Finally, terminate the program
  delete g4MPI;
  delete runManager;
}


* Notes about session shell:

LAM/MPI users can use "G4tcsh" as an interactive session shell.
For other users (Open MPI/MPICH2), plese use G4csh (default).


How to use:
-----------
* MPI runtime rnvironment

1. Make hosts/cluster configuration of your MPI environment.

2. Launch MPI runtime environment, typically executing 
   lamboot (LAM) / mpdboot (MPICH2).

* How to run
For example,

> mpiexec -n # <your application>

Instead, "mpirun" command is more convenient for LAM users.


MPI G4UI commands:
------------------
G4UI commands handling the G4MPI interface are placed in /mpi/.

Command directory path : /mpi/

Guidance :
MPI control commands

 Sub-directories :
 Commands :
   verbose *        Set verbose level.
   status *         Show mpi status.
   execute *        Execute a macro file. (=/control/execute)
   beamOn *         Start a parallel run w/ thread.
   .beamOn *        Start a parallel run w/o thread. (=/run/beamOn)
   masterWeight *   Set weight for master node.
   showSeeds *      Show seeds of MPI nodes.
   setMasterSeed *  Set a master seed for the seed generator.
   setSeed *        Set a seed for a specified node.

* Notes:
While "/run/beamOn" is executed in foreground, the "/mpi/beamOn" 
command invokes beam-on in background, so you can input UI commands
even while event processing.

The original "/control/execute" and "/run/beamOn" are overwritten 
with "/mpi/execute" and "/mpi/.beamOn" commands, customized for 
the MPI interface.


Examples
========
There are a couple of examples for Geant4 MPI applications.

In some cases, you need to set some additional environment variables
for running examples:

- G4LEVELGAMMADATA : directory path for the data of 
                     "low energy electromagnetic processes".

For running ROOT applications, 

- ROOTSYS         : root path of the ROOT package
- LD_LIBRARY_PATH : add ROOT library path, such as $(ROOTSYS)/lib/root


[exMPI01]
A simple application.

* Geometry     : chamber / calorimeter
* Primary      : particle gun (200 MeV electron as default)
* Physics List : standard EM

- Particles are transported in a geometry without any scoring.
- Learn how to parallelize your G4 session.


[exMPI02] (ROOT application)
An example of dosimetry in a water phantom.

* Geometry     : water phantom
* Primary      : particle gun (200 MeV proton as default)
* Physics List : QGSP_BIC

- Score dose distribution in a water phantom.
- Learn how to paralleize your applications.
- Create a ROOT file containing histograms in each node.
  * treat multiple outputs properly:
    each slave node generate a ROOT file, whose file name should
    be different from each other.