[1337] | 1 | $Id: README,v 1.3 2010/05/18 06:09:09 kmura Exp $ |
---|
[807] | 2 | ========================================================================== |
---|
| 3 | Geant4 MPI Interface |
---|
| 4 | |
---|
| 5 | Author: |
---|
| 6 | Koichi Murakami (KEK) / Koichi.Murakami@kek.jp |
---|
| 7 | ========================================================================== |
---|
| 8 | |
---|
| 9 | About the interface |
---|
| 10 | =================== |
---|
| 11 | G4MPI is a native interface with MPI libraries. The directory contains |
---|
| 12 | a Geant4 UI library and a couple of parallelized examples. |
---|
| 13 | Using this interface, users applications can be parllelized with |
---|
[1337] | 14 | different MPI compliant libraries, such as OpenMPI, LAM/MPI, MPICH2, |
---|
[807] | 15 | and so on. |
---|
| 16 | |
---|
| 17 | System Requirements: |
---|
| 18 | ------------------- |
---|
| 19 | * Platform |
---|
| 20 | |
---|
| 21 | - Linux (32bit/64bit) |
---|
| 22 | - MacOS (Tiger) |
---|
| 23 | |
---|
| 24 | |
---|
| 25 | * MPI implementation |
---|
| 26 | |
---|
[1337] | 27 | - Open MPI (1.4.2) |
---|
| 28 | http://www.open-mpi.org/ |
---|
| 29 | |
---|
| 30 | - LAM/MPI (7.1.3) (development is terminated. maintenance only) |
---|
[807] | 31 | http://www.lam-mpi.org/ |
---|
| 32 | |
---|
[1337] | 33 | - MPICH2 (1.2.1p1) |
---|
[807] | 34 | http://www.mcs.anl.gov/research/projects/mpich2/ |
---|
| 35 | |
---|
| 36 | * Optional |
---|
| 37 | - ROOT for histogramming/analysis |
---|
| 38 | |
---|
| 39 | |
---|
| 40 | G4MPI UI library |
---|
| 41 | ---------------- |
---|
| 42 | G4MPI UI library is provided in the "mpi_interface" directory. |
---|
[1337] | 43 | Start with building the G4MPI UI library according to the instruction |
---|
[807] | 44 | in the directory. |
---|
| 45 | |
---|
| 46 | |
---|
| 47 | How to make parallel applications: |
---|
| 48 | ---------------------------------- |
---|
| 49 | * An example of a main program: |
---|
| 50 | |
---|
| 51 | #include "G4MPImanager.hh" |
---|
| 52 | #include "G4MPIsession.hh" |
---|
| 53 | |
---|
| 54 | int main(int argc,char** argv) |
---|
| 55 | { |
---|
| 56 | // At first, G4MPImanager/G4MPIsession should be created. |
---|
| 57 | G4MPImanager* g4MPI= new G4MPImanager(argc,argv); |
---|
| 58 | |
---|
| 59 | // MPI session (G4MPIsession) instead of G4UIterminal |
---|
| 60 | G4MPIsession* session= g4MPI-> GetMPIsession(); |
---|
| 61 | |
---|
| 62 | // user application setting |
---|
| 63 | G4RunManager* runManager= new G4RunManager(); |
---|
| 64 | |
---|
| 65 | // .... |
---|
| 66 | |
---|
| 67 | // After user application setting, just start a MPI session. |
---|
| 68 | // MPIsession treats both interactive and batch modes. |
---|
| 69 | session-> SessionStart(); |
---|
| 70 | |
---|
| 71 | // Finally, terminate the program |
---|
| 72 | delete g4MPI; |
---|
| 73 | delete runManager; |
---|
| 74 | } |
---|
| 75 | |
---|
| 76 | |
---|
| 77 | * Notes about session shell: |
---|
| 78 | |
---|
| 79 | LAM/MPI users can use "G4tcsh" as an interactive session shell. |
---|
| 80 | For other users (Open MPI/MPICH2), plese use G4csh (default). |
---|
| 81 | |
---|
[1337] | 82 | In case of OpenMPI, LD_LIBRARY_PATH for OpenMPI runtime libraries |
---|
| 83 | should be set at run time. Alternatively, you can add this path |
---|
| 84 | to the dynamic linker configuration using `ldconfig`. |
---|
| 85 | (needs sys-admin authorization) |
---|
[807] | 86 | |
---|
[1337] | 87 | |
---|
[807] | 88 | How to use: |
---|
| 89 | ----------- |
---|
| 90 | * MPI runtime rnvironment |
---|
| 91 | |
---|
| 92 | 1. Make hosts/cluster configuration of your MPI environment. |
---|
| 93 | |
---|
[1337] | 94 | 2. Launch MPI runtime environment, typically executing |
---|
[807] | 95 | lamboot (LAM) / mpdboot (MPICH2). |
---|
| 96 | |
---|
| 97 | * How to run |
---|
| 98 | For example, |
---|
| 99 | |
---|
| 100 | > mpiexec -n # <your application> |
---|
| 101 | |
---|
| 102 | Instead, "mpirun" command is more convenient for LAM users. |
---|
| 103 | |
---|
| 104 | |
---|
| 105 | MPI G4UI commands: |
---|
| 106 | ------------------ |
---|
| 107 | G4UI commands handling the G4MPI interface are placed in /mpi/. |
---|
| 108 | |
---|
| 109 | Command directory path : /mpi/ |
---|
| 110 | |
---|
| 111 | Guidance : |
---|
| 112 | MPI control commands |
---|
| 113 | |
---|
| 114 | Sub-directories : |
---|
| 115 | Commands : |
---|
| 116 | verbose * Set verbose level. |
---|
| 117 | status * Show mpi status. |
---|
| 118 | execute * Execute a macro file. (=/control/execute) |
---|
| 119 | beamOn * Start a parallel run w/ thread. |
---|
[1337] | 120 | .beamOn * Start a parallel run w/o thread. |
---|
[807] | 121 | masterWeight * Set weight for master node. |
---|
[1337] | 122 | wait * Wait until beamOn-s on all nodes are done. (batch mode only) |
---|
[807] | 123 | showSeeds * Show seeds of MPI nodes. |
---|
| 124 | setMasterSeed * Set a master seed for the seed generator. |
---|
| 125 | setSeed * Set a seed for a specified node. |
---|
| 126 | |
---|
| 127 | * Notes: |
---|
[1337] | 128 | While "/run/beamOn" and "/mpi/beamOn" commands invoke beam-on in background, |
---|
| 129 | so you can input UI commands even while event processing. |
---|
[807] | 130 | |
---|
[1337] | 131 | The original "/control/execute" and "/run/beamOn" are overwritten |
---|
| 132 | with "/mpi/execute" and "/mpi/beamOn" commands respectively, |
---|
| 133 | customized for the MPI interface. |
---|
[807] | 134 | |
---|
| 135 | |
---|
| 136 | Examples |
---|
| 137 | ======== |
---|
| 138 | There are a couple of examples for Geant4 MPI applications. |
---|
| 139 | |
---|
| 140 | In some cases, you need to set some additional environment variables |
---|
| 141 | for running examples: |
---|
| 142 | |
---|
[1337] | 143 | - G4LEVELGAMMADATA : directory path for the data of |
---|
[807] | 144 | "low energy electromagnetic processes". |
---|
| 145 | |
---|
[1337] | 146 | For running ROOT applications, |
---|
[807] | 147 | |
---|
| 148 | - ROOTSYS : root path of the ROOT package |
---|
| 149 | - LD_LIBRARY_PATH : add ROOT library path, such as $(ROOTSYS)/lib/root |
---|
| 150 | |
---|
| 151 | |
---|
| 152 | [exMPI01] |
---|
| 153 | A simple application. |
---|
| 154 | |
---|
| 155 | * Geometry : chamber / calorimeter |
---|
| 156 | * Primary : particle gun (200 MeV electron as default) |
---|
| 157 | * Physics List : standard EM |
---|
| 158 | |
---|
| 159 | - Particles are transported in a geometry without any scoring. |
---|
| 160 | - Learn how to parallelize your G4 session. |
---|
| 161 | |
---|
| 162 | |
---|
| 163 | [exMPI02] (ROOT application) |
---|
| 164 | An example of dosimetry in a water phantom. |
---|
| 165 | |
---|
| 166 | * Geometry : water phantom |
---|
| 167 | * Primary : particle gun (200 MeV proton as default) |
---|
| 168 | * Physics List : QGSP_BIC |
---|
| 169 | |
---|
| 170 | - Score dose distribution in a water phantom. |
---|
| 171 | - Learn how to paralleize your applications. |
---|
| 172 | - Create a ROOT file containing histograms in each node. |
---|
| 173 | * treat multiple outputs properly: |
---|
| 174 | each slave node generate a ROOT file, whose file name should |
---|
[1337] | 175 | be different from each other. |
---|
[807] | 176 | |
---|