|
|
 |
1. How to access mphase cluster:
You can ssh (secure shell) to mphase2 from the Rutgers subnets,
for example, user jerry can do it as follows:
ssh jerry@mphase2
If you do it at first time, type "yes" to accept RSA1 key, then type-in your
password at the login prompt. Similarly, you can sftp (secure ftp) to the same host:
sftp jerry@mphase2
then use commands "get" and "put". If you don't have ssh/sftp client on your
machine, you can download free versions of it from www.openssh.org
or www.ssh.com
2. How to submit your job to the queue system:
The submit scripts, batch_run_simple.sh, batch_run.sh and mpi_run.sh can be downloaded from Sample Queue Scripts then modified for your cases and submitted to the queue system with
qsub command, for example:
qsub batch_run.sh
The simplest batch script is batch_run_simple.sh. It can be used for any generic serial job.
However if your application writes/reads a lot of data (I/O to files), you'd get much
better performance utilizing a local node /tmp directory. Use batch_run.sh submit script in such a case.
For the users who got used to LSF on RCG cluster, I wrote simple wrappers, bsub, bjobs and bkill,
to immitate such commands on LSF; using bsub, you can submit a binary executable to SGE queue system.
3. How to compile your source code:
Your source codes written in Fortran and C can be compiled with GNU, Intel and PGI compilers.
To compile your codes with the GNU-3.3.5 compilers, proceed as follows:
gcc -o mycode.x mycode.c
g77 -o mycode.x mycode.f
To compile your codes with the Intel compilers, use the following commands:
icc -o mycode.x mycode.c
ifort -o mycode.x mycode.f
To compile your codes with the PGI compilers:
pgcc -o mycode.x mycode.c
pgf77 -o mycode.x mycode.f
pgf90 -o mycode.x mycode.f
4. How to compile your code with MPI
(MPICH 1.2.7):
First, you need to modify the PATH environmant variable to include the full path to the directories containing MPI commands and libraries.
For example, to use MPI with GNU gcc-3.3.5 on mphase2, include in your .bashrc or .login the following two lines:
export MPIR_HOME=/usr/local/mpich-1.2.7/gcc
export PATH=$PATH:$MPIR_HOME/bin:$MPIR_HOME/include:$MPIR_HOME/lib
To use MPI with Intel 9.0 compilers, change the MPIR_HOME above for
MPIR_HOME=/usr/local/mpich-1.2.7/ia32
To use MPI with PGI compilers, change the MPIR_HOME above for
MPIR_HOME=/usr/local/mpich-1.2.7/pgi6.0
After the environment variables above are set, you can compile your code using
mpicc (C-codes), mpiCC (C++ codes), mpif77 (f77 codes) or mpif90 (f90 codes).
For example, to compile a C code with MPI:
mpicc -o mycode.x mycode.c
To compile a fortan77 code with MPI:
mpif77 -o mycode.x mycode.f
To compile a fortan90 code with MPI:
mpif90 -o mycode.x mycode.f
(MPICH 2):
On some of the newest Engineering clusters, there is MPICH-2 installed.
To verify if MPICH 2 is available, run command dpkg -l mpich2-* and see if it gives you a list of installed packages related to
MPICH 2.
You need to set the PATH environment variable to include
the directories containing MPI 2 commands and libraries.
If you intend using MPI with GNU (GCC and GFORTRAN) compilers, include in your .bashrc the following lines:
export MPIR_HOME=/usr/local/mpich2_smpd/gnu
export PATH=$PATH:$MPIR_HOME/bin:$MPIR_HOME/include:$MPIR_HOME/lib
To use MPI with PGI compilers, change the MPIR_HOME above for
MPIR_HOME=/usr/local/mpich2_smpd/pgi
Compile your code with mpicc, mpif77 or mpif90 as above, in the case with
MPICH 1.2.7.
5. How to run parallel MPI jobs
(MPICH 1.2.7):
Script batch_run.sh is designed for single processor jobs; mpi_run.sh is for multi-processor MPI.
In the parallel environment variable of mpi_run.sh, -pe, you can specify one of the two
subclusters you would use:
mpich_intel (Intel nodes, 15 CPUs maximum), mpich_amd (AMD Athlon nodes, 8 CPUs maximum),
mpich_samd (Dual-AMD Atlon, 86 CPUs maximum):
Depending on the compiler you use, either gcc3.3.5 or Intel 9.0, you need to Specify MPIR_HOME in the script:
#$ -v MPIR_HOME=/usr/local/mpich-1.2.7/gcc
or
#$ -v MPIR_HOME=/usr/local/mpich-1.2.7/ia32
or
#$ -v MPIR_HOME=/usr/local/mpich-1.2.7/pgi6.0
(MPICH 2):
You need to create a new file in your home directory .smpd containing a passphrase for communication between MPI management daemons. The content of file .smpd, for example, can be as follows:
phrase=secret1
The file can be the same for all MPICH 2 jobs you'll be running on the cluster.
Download a template submit script for MPICH 2 jobs and modify it for your case,
mpich2_gnu_script.sh or mpich2_pgi_script.sh.
Check for available parallel environments on the system:
qconf -spl
Note, you need to match the requested parallel environment with the type of
MPICH 2 you used to compile your code, GNU or PGI, for example.
6. How to monitor your running job:
To monitor running jobs, you can execute command:
qstat
To check the status of the queue system, run:
qstat -f
7. How to delete your submitted job from the queue:
To delete the submitted job from the queue:
qdel jobID
For running jobs, use the force flag, "-f":
qdel -f jobID
8. How to check accounting for your finished jobs:
Use command qacct. For example, if you want to check the accounting for the job with jobnumber 34:
qacct -j 34
To check accounting for all jobs that belonged to user jerry and finished during the past 2 days:
qacct -j -o jerry -d 2
9. How to recover your accidently deleted files:
We run incremental daily backups from Monday through Friday. If you need to restore something from the backups,
please e-mail to Alexei (alexei@soemail.rutgers.edu), specifying the file name, the directory where the file was located, the date
when you lost it and the date when you modified it the last time.
 For any technical questions contact Admin at:
alexei@soemail.rutgers.edu
To the top
|
|
© 2003 All Rights Reserved.
Rutgers, The State University of New Jersey.
|
|
