A brief guide to use the MIMS-SMP

Contents

日本語バージョンへ飛ぶ

Note: The symbol "$" is the prompt symbol.


Overview

MIMS SMP consists of four computation servers and one login server.

Hostname Description
mims-smp 48 core cpus and 1.5TB RAM

mims-gpu1,

mims-gpu2

GPGPU TESLA K80
mims-cpu1 Intel Xeon E5-2637v3 3.5 GHz

Flow of use

  1. Login to mims-gate by SSH, and then login to mims-smp by SSH.
  2. Prepare execution files and a ".q" file on mims-smp.
  3. Submit your computation job by qsub command on mims-smp.
  4. Logout and wait for completing the job.

Login

Login to mims-gate by ssh, and then login to mims-smp by ssh.

For example, if you are working on a Unix-like environment, launch a terminal emulator and do the followings:

$ ssh username@mims-gate.mind.meiji.ac.jp

$ ssh mims-smp

Here "username" above should be replaced by your own username. The first command should be done on your server, and the second command should be done on mims-gate.

Notes:

  1. It is prohibited to login mims-smp from other than mims-gate.
  2. It is prohibited to login mims-gate from abroad.
  3. A Windows user may have to install an SSH client software since Microsoft Windows does not provide with it by default. See here for details of installation.

Logout

Enter the exit command.

$ exit


Changing your password

You can change your password by the passwd command:

$ passwd

After entering the current password of yours, you will be required to enter the new password twice.

Notes:

  1. Passwords must be secure.
  2. Especially, the password for mims-gate must be absolutely secure: mims-gate is open to the world.

File transfer

sftp and scp are available on both mims-gate and mims-smp. Note that conmutation servers share the home directory of mims-smp. So any file in mims-smp can be seen from other computation servers.

Copying a file from client to mims-smp

[user's PC]$ scp filename username@mims-gate.mind.meiji.ac.jp:

[user's PC]$ ssh username@mims-gate.mind.meiji.ac.jp

[mims-gate]$ scp filename mims-smp:

Replace filename and username by the name of file and the user ID, respectively.

The first line copies filename from client to the home directory of mims-gate. The second line lets you login to mims-gate. The third line copies filename to the home directory of mims-smp.

 

Copying a file from mims-smp to client

[mims-smp ]$ scp filename mims-gate:

[user's PC]$ scp username@mims-gate:filename .

The first line copies filename from mims-smp to mims-gate. The second line copies filename from mims-gate to client (Don't miss the period at the end of line).


Submitting your job

Note: When you do any computation by MIMS-SMP, please submit your job by using qsub command.

We would greatly appreciate your cooperation.

Submission flow

  1. Prepare your programme.
  2. Prepare a batch file.
  3. Do the following and wait (You may logout from mims-smp after entering the command):

[mims-smp ]$ qsub < name-of-the-batch-file

(Substitute the name of your batch file to name-of-the-batch-file.)

You can check submnitted jobs by the qstat command.

$ qstat

Also, you can check the trace of your job by tracejob command.

$ tracejob jobid

(Substitute the job ID to jobid)

 

Sample

Suppose that we save the following C code as hello.c:

#include <stdio.h>

main()

{

#pragma omp parallel

printf("Hello world\n");

}

 

Suppose the path to hello.c is: $HOME/work/hello.c

Compile it

$ cd $HOME/work

$ icc -fopenmp hello.c

 

Save the following script as hello.q

#!/bin/sh

#PBS -N helloworld

#PBS -j oe

#PBS -l ncpus=12

#PBS -q low

$HOME/work/a.out

 

Submit the job:

$ qsub < hello.q

After a while, the program will run. The output will be written in the file helloworld.o**, where ** is the job ID.

Options of qsub
Option Description
-N <comment> Set the name of job to <comment>
-j oe Create output and error files
-l ncpus=<num> Set the number of cores to <num>
-l nodes=<num> Set the number of nodes to <num>
-q <queue_name> Select the type of queue: low, mid, or hi

You can specify the options above in the batch file: The line starting by "#PBS" specifies the option of qsub.

See "man qsub" for details (see also "man qstat", "man tracejob").

Type of queue
Queue # of cores Memory CPU time Computation node note
low 12 370GB unlimited mims-smp DefaultQueue
mid 24 750GB unlimited mims-smp  
hi 48 1.5TB unlimited mims-smp  
cpu 8 64GB unlimited mims-cpu1  
gpu 16 64GB unlimited

mims-gpu1,

mims-gpu2

For GPGPU

 

 

Samples : mid, hi


Numerics

Intel C++/Fortran Compiler(on mims-smp, mims-cpu1, and mims-gpu1&2)

Intel C++ and Fortran Compilers are available as well as Intel(R) Math Kernel Library (MKL).

Intel(R) MKL supports BLAS, LAPACK, and some other routines for numerical computation including FFT. Also, MKL provides with CBLAS and LAPACKE interfaces. See documents for details.

When you compile a programme with MKL library, set the compiler option "-mkl";

$ ifort -mkl yourprogramme.f90

or

$ icc -mkl yourprogramme.c

etc.

Note: If your programme is written in C/C++, "mkl.h" must be included.

 

Click here to download samples (three ways to solve the Poisson equation using CBLAS and LAPACKE).

PGI C/C++/Fortran Compiler(mims-gpu1&2)

PGI compilers are available on mims-gpu1 and mims-gpu2.

For Fortran

[mims-gpu1 ]$ pgfortran yourprogramme.f90

For C

[mims-gpu1 ]$ pgcc yourprogramme.c

For C++

[mims-gpu1 ]$ pgCC yourprogramme.cpp


Plotting tool (Gnuplot on mims-smp)

Gnuplot 5.0.0 is available on mims-smp.

[mims-smp ]$ gnuplot

Terminals

The default terminal is "unknown". So plot command draws no piture. Choose your favorite terminal by "set terminal" command to output a picuture to a file. You can see available terminals by the following command:

gnuplot> set terminal

If you are using X11 forwarding, you can view

[mims-smp ]$ eog filename

[mims-smp ]$ evince filename


Q&A

Q. How to fix "/bin/sh^M: bad interpreter: No such file or directory" error?

A. This error occurs due to EOF code. Typically, this occurs when you are going to submit a ".q" file created on Windows PC. EOF code in ".q" file should be Unix style.

I suggest two solutions:

[mims-smp ]$ tr -d \\r < win.q > unix.q

 

Q. How can I use X11 forwarding on SSH connection?

A. For users of Unix-like environment: Use ssh command with "-XY" option.

[user's PC]$ ssh -XY username@mims-gate.mind.meiji.ac.jp

[mims-gate]$ ssh -XY mims-smp

For Windows users: Please see a manual of PuTTY.

 

Q. Is there a time limit of a computation job?

A. There is no limit now, but maybe in future.

You can set a time limit by yourself. Add the following line in ".q" script:

#PBS -l walltime=01:23:45

In this case, the job will stop and be deleted if the time lapse exceeds 1 hour 23 minutes 45 seconds. "01:23:45" means the time in the format "HH:MM:SS". Please replace it by your own choice.

 

Q. I want an email notification when my job is done.

A. Add the following line in ".q" script:

#PBS -m abe -M your-email-address

 

Three characters following "-m" are:

a: Send an email if the job is aborted.

b: Send an email if the job begins.

e: Send an email if the job terminates.

Please replace "your-email-address" by the email address of yours.


Written by Tomoyuki Miyaji

2015-11-02