MIMS Shared Memory Processor

Overview

MIMS SMP consists of four computation servers, two computation servers for GPGPU and one login server.

Computing server (mims-smp)
HPE ProLiant DL580 Gen10 server
CPU96core/Intel Xeon-Platinum 8260 (2.4GHz/24-core) x 4
Memory1.5TB/32GB DIMM x 48
GPUHPE NVIDIA Tesla V100 PCIe 32GB Computational Accelerator
Storage約100TB
Software・Intel PS Composer 2 Edu 3yr LTU
・Altair PBS Pro 1 Sckt 3yr Perp LTU

 
HPE ProLiant DL380 Gen10 server
CPU 16core/Xeon-Silver 4208 (2.1GHz/8-core) x 2
Memory 256GB/32GB DIMM x 8
GPUNVIDIA Quadro RTX 6000 Graphics Accelerator
OS Cent OS
Software Open PBS Pro
Performance Single-precision: 16.3TFlops

Flow of use

  • 1. Login to mims-gate by SSH, and then login to mims-smp by SSH.
  • 2. Prepare execution files and a ".q" file on mims-smp.
  • 3. Submit your computation job by qsub command on mims-smp.
  • 4. Logout and wait for completing the job.

Login / Logout

Login

Login to mims-gate by ssh, and then login to mims-smp by ssh.
For example, if you are working on a Unix-like environment, launch a terminal emulator and do the followings:

  [user's PC ] $ ssh username@mims-gate.mind.meiji.ac.jp
  [mims-gate] $ ssh mims-smp

Here username above should be replaced by your own username. The first command should be done on your server, and the second command should be done on mims-gate.

Note:

  • 1. It is prohibited to login mims-smp from other than mims-gate.
  • 2. It is prohibited to login mims-gate from abroad.
  • 3. A Windows user may have to install an SSH client software since Microsoft Windows does not provide with it by default.
  • 4. See here for details of installation.
 
Logout

Enter the exit command.

  [mims-smp] $ exit
  [mims-gate] $ exit

Changing your password

You can change your password by the passwd command:

  $ passwd

After entering the current password of yours, you will be required to enter the new password twice.


Note:

  • 1. Passwords must be secure.
  • 2. Especially, the password for mims-gate must be absolutely secure:
     mims-gate is open to the world.

File transfer

sftp and scp are available on both mims-gate and mims-smp. Note that conmutation servers share the home directory of mims-smp. So any file in mims-smp can be seen from other computation servers.

Copying a file from client to mims-smp

  [user's PC ] $ scp filename username@mims-gate.mind.meiji.ac.jp:
  [user's PC ] $ ssh username@mims-gate.mind.meiji.ac.jp
  [mims-gate] $ scp filename mims-smp:

Replace filename and username by the name of file and the user ID, respectively.

The first line copies filename from client to the home directory of mims-gate. The second line lets you login to mims-gate. The third line copies filename to the home directory of mims-smp.

Copying a file from mims-smp to client

  [mims-smp ] $ scp filename mims-gate:
  [user's PC] $ scp username@mims-gate:filename .

The first line copies filename from mims-smp to mims-gate. The second line copies filename from mims-gate to client (Don't miss the period at the end of line).

Submitting your job

Note:
When you do any computation by MIMS-SMP, please submit your job by using qsub command.
We would greatly appreciate your cooperation. 

Submission flow
  • 1. Prepare your programme.
  • 2. Prepare a batch file.
  • 3. Do the following and wait (You may logout from mims-smp after entering the command):


   [mims-smp] $ qsub < name-of-the-batch-file
(Substitute the name of your batch file to name-of-the-batch-file.)

You can check submnitted jobs by the qstat command.
   [mims-smp] $ qstat

Also, you can check the trace of your job by tracejob command.

   [mims-smp]$ tracejob jobid
(Substitute the job ID to jobid)

Sample

Suppose that we save the following C code as hello.c:
  #include  
  main()
  {
  #pragma omp parallel
  printf("Hello world\n");
  }

Suppose the path to hello.c is: $HOME/work/hello.c

Compile it
  $ cd $HOME/work
  $ icc -fopenmp hello.c

Save the following script as hello.q
  #!/bin/sh
  #PBS -N helloworld
  #PBS -j oe
  #PBS -l ncpus=12
  #PBS -q low
  $HOME/work/a.out

Submit the job:
  $ qsub < hello.q

After a while, the program will run. The output will be written in the file helloworld.o**, where ** is the job ID.


◆ Options of qsub

OptionDescription
-N <comment>Set the name of job to <comment>
-j oeCreate output and error files
-l mem=<num>Set the limit of memory usage
-l ncpus=<num>Set the number of cores to <num>
-l walltime=HH:MM:SSSet the limit of walltime
-q <queue_name>Select the type of queue: low, mid, hi, cpu, or gpu


You can specify the options above in the batch file: The line starting by "#PBS" specifies the option of qsub.

See "man qsub" for details (see also "man qstat", "man tracejob" ).

◆ Type of queue

Queue # of cores Memory GPU CPU time Computation
node
low 24 370GB unlimited mims—smp
mid 48 750GB unlimited mims—smp
hi 96 1.5TB unlimited mims—smp
gpu 8 128GB 1 unlimited mims—smp
gpu2 16 256GB 1 unlimited mims-smp-gpu0


 We have introduced the computing servers for GPGPU (mims-smpgpu0) on July 2021.
 Specify the queue as "gpu2".

 
Samples :

Numerics

Intel Compiler
(on mims-smp)
Intel PS Composer 2 Edu 3yr LTU is available.


Note:
When you compile a programme with MKL library, set the path to the MKL library.
/opt/intel/mkl/lib/intel64
 

If your programme is written in C/C++, "mkl.h" must be included.

Click here to download samples.

   (Three ways to solve the Poisson equation using CBLAS and LAPACKE)

Plotting tool
(Gnuplot on mims-smp)

Gnuplot 5.0.0 is available on mims-smp.
  [mims-smp] $ gnuplot

Terminals

The default terminal is "unknown". So plot command draws no piture. Choose your favorite terminal by "set terminal" command to output a picuture to a file. You can see available terminals by the following command:

  gnuplot> set terminal
 

If you are using X11 forwarding, you can view

 ● jpeg, png, or gif files by "eog" command.
 ● ps, eps, or pdf files by "evince" command.
 

  [mims-smp] $ eog filename
  [mims-smp] $ evince filename

Q&A

Q.
How to fix "/bin/sh^M: bad interpreter: No such file or directory" error?

A.

This error occurs due to EOF code. Typically, this occurs when you are going to submit a ".q" file created on Windows PC. EOF code in ".q" file should be Unix style.

I suggest two solutions:

 ● Prepare your ".q" file with Unix style EOF code. A sophiosticated text editor has functinons to choose EOF code.
 ●Use tr command for string processing.Use tr command for string processing.
  [mims-smp] $ tr -d \\r < win.q > unix.q

Q. How can I use X11 forwarding on SSH connection?

A.

For users of Unix-like environment: Use ssh command with "-XY" option.

  [user's PC ] $ ssh -XY username@mims-gate.mind.meiji.ac.jp
  [mims-gate] $ ssh -XY mims-smp  

For Windows users:
Please see a manual of PuTTY.

Q. Is there a time limit of a computation job?

A.

There is no limit now, but maybe in future.

You can set a time limit by yourself. Add the following line in ".q" script:
  #PBS -l walltime=01:23:45

In this case, the job will stop and be deleted if the time lapse exceeds 1 hour 23 minutes 45 seconds. "01:23:45" means the time in the format "HH:MM:SS". Please replace it by your own choice.

Q. I want an email notification when my job is done.

A.

Add the following line in ".q" script:
  #PBS -m abe -M your-email-address

Three characters following "-m" are:
     a: Send an email if the job is aborted.
     b: Send an email if the job begins.
     e: Send an email if the job terminates.

Please replace "your-email-address" by the email address of yours

Support

If you have any question, please don't hesitate to contact the support room.

Name of representative:SHIRAISHI Masashi (Associate Professor)
Email:***@meiji.ac.jp (replace *** by mimssmp)