Skip to content
Snippets Groups Projects
Commit 90762624 authored by Michael Blaschek's avatar Michael Blaschek :bicyclist:
Browse files

Update VSC.md

parent eec4369a
No related branches found
No related tags found
No related merge requests found
...@@ -3,46 +3,208 @@ ...@@ -3,46 +3,208 @@
[VSC](https://vsc.ac.at/home/) [VSC](https://vsc.ac.at/home/)
[VSC-Wiki](https://wiki.vsc.ac.at) [VSC-Wiki](https://wiki.vsc.ac.at)
We have the privilege to be part of the VSC and have private nodes at VSC-4 (currently operational since 2019). [[__TOC__]]
To get access write MB and request an account on VSC-4. Please send the follogin details: We have the privilege to be part of the VSC and have private nodes at VSC-4 (since 2020) and VSC-3 (since 2014)
- Name
- Email
- Mobile number (required)
- Purpose / Group
We can then give you access to Access is primarily via SSH:
`ssh vsc4.vsc.ac.at` ```
ssh user@vsc4.vsc.ac.at
ssh user@vsc3.vsc.ac.at
```
Please follow some connection instruction on the [wiki](https://wiki.vsc.ac.at) or similar to all other servers (e.g. [SRVX1](SRVX1.md)). Please follow some connection instruction on the [wiki](https://wiki.vsc.ac.at) which is similar to all other servers (e.g. [SRVX1](SRVX1.md)).
The VSC is only available from within the UNINET (VPN, ...).
We have private nodes at our disposal and in order for you to use these you need to specify the correct account in the jobs you submit to the queueing system (SLURM). The correct information will be given to you in the email. This is very similar to Jet. We have private nodes at our disposal and in order for you to use these you need to specify the correct account in the jobs you submit to the queueing system (SLURM). The correct information will be given to you in the registration email.
## Node Information: ## Node Information VSC-4
``` ```
CPU model: Intel(R) Xeon(R) Platinum 8174 CPU @ 3.10GHz CPU model: Intel(R) Xeon(R) Platinum 8174 CPU @ 3.10GHz
2 CPU, 24 physical cores per CPU, total 96 logical CPU units 2 CPU, 24 physical cores per CPU, total 96 logical CPU units
378 GB Memory
```
## Node Information VSC-3
```
CPU model: Intel(R) Xeon(R) CPU E5-2650 v2 @ 2.60GHz
2 CPU, 8 physical cores per CPU, total 32 logical CPU units
126 GB Memory
``` ```
### Run time limits
On VSC-3 we have a max runtime of 10 days for the private Queue. The normal queues have 3 days. the devel only 10 min (for testing)
```
sacctmgr show qos format=name%20s,priority,grpnodes,maxwall,description%40s
Name Priority GrpNodes MaxWall Descr
-------------------- ---------- -------- ----------- ----------------------------------------
p70653_0128 100000 10 10-00:00:00 priv nodes univie meteorologie
normal_0064 2000 1286 3-00:00:00 all user
normal_0256 2000 6 3-00:00:00 all user
normal_0128 2000 11 3-00:00:00 all user
devel_0128 5000000 10 00:10:00 for developing and testing codes
```
on VSC-4 accordingly.
```
sacctmgr show qos format=name%20s,priority,grpnodes,maxwall,description%40s
Name Priority GrpNodes MaxWall Descr
-------------------- ---------- -------- ----------- ----------------------------------------
p71386_0384 100000 10-00:00:00 private nodes haimberger
long 1000 10-00:00:00 long running jobs on vsc-4
fast_vsc4 1000000 3-00:00:00 high priority access
idle_0096 1 1-00:00:00 vsc-4 idle nodes
idle_0384 1 1-00:00:00 vsc-4 idle nodes
idle_0768 1 1-00:00:00 vsc-4 idle nodes
mem_0096 1000 3-00:00:00 vsc-4 regular nodes with 96 gb of memory
mem_0384 1000 3-00:00:00 vsc-4 regular nodes with 384 gb of memo+
mem_0768 1000 3-00:00:00 vsc-4 regular nodes with 768 gb of memo+
```
SLURM allows for setting a run time limit below the default QOS's run time limit. After the specified time is elapsed, the job is killed:
```
#SBATCH --time=<time>
```
Acceptable time formats include `minutes`, `minutes:seconds`, `hours:minutes:seconds`, `days-hours`, `days-hours:minutes` and `days-hours:minutes:seconds`.
## Example Job ## Example Job
:construction: - [VSC Wiki Slurm](https://wiki.vsc.ac.at/doku.php?id=doku:slurm)
- [VSC Wiki private Queue](https://wiki.vsc.ac.at/doku.php?id=doku:vsc3_queue)
## Software
The VSC use the same software system as Jet and have environmental modules available to the user: ### Example Job on VSC-3
We have 16 CPUs per Node. In order to fill:
- `--partition=mem_xxxx` (per email)
- `--qos=xxxxxx` (see below)
- `--account=xxxxxx` (see below)
The core hours will be charged to the specified account. If not specified, the default account will be used.
on VSC-3 our account is called:
```
sacctmgr show user `id -u` withassoc format=user,defaultaccount,account,qos%40s,defaultqos%20s
User Def Acct Account QOS Def QOS
---------- ---------- ---------- ---------------------------------------- --------------------
71633 p70653 p70653 devel_0128,p70653_0128 p70653_0128
```
Put this in the Job file:
```
#!/bin/bash
#
#SBATCH -J TEST_JOB
#SBATCH -N 2
#SBATCH --ntasks-per-node=16
#SBATCH --ntasks-per-core=1
#SBATCH --mail-type=BEGIN # first have to state the type of event to occur
#SBATCH --mail-user=<email@address.at> # and then your email address
#SBATCH --partition=mem_xxxx
#SBATCH --qos=p70653_0128
#SBATCH --account=p70653
#SBATCH --time=<time>
# when srun is used, you need to set (Different from Jet):
<srun -l -N2 -n32 a.out >
# or
<mpirun -np 32 a.out>
```
* **-J** job name
* **-N** number of nodes requested (16 cores per node available)
* **-n, --ntasks=<number>** specifies the number of tasks to run,
* **--ntasks-per-node** number of processes run in parallel on a single node
* **--ntasks-per-core** number of tasks a single core should work on
* **srun** is an alternative command to **mpirun**. It provides direct access to SLURM inherent variables and settings.
* **-l** adds task-specific labels to the beginning of all output lines.
* **--mail-type** sends an email at specific events. The SLURM doku lists the following valid mail-type values: *"BEGIN, END, FAIL, REQUEUE, ALL (equivalent to BEGIN, END, FAIL and REQUEUE), TIME_LIMIT, TIME_LIMIT_90 (reached 90 percent of time limit), TIME_LIMIT_80 (reached 80 percent of time limit), and TIME_LIMIT_50 (reached 50 percent of time limit). Multiple type values may be specified in a comma separated list."* [cited from the SLURM doku](http://slurm.schedmd.com)
* **--mail-user** sends an email to this address
``` ```
module av [username@l31 ~]$ sbatch check.slrm # to submit the job
[username@l31 ~]$ squeue -u `whoami` # to check the status of own jobs
[username@l31 ~]$ scancel JOBID # for premature removal, where JOBID
# is obtained from the previous command
``` ```
will show you all available modules (much more than on jet) ### Example Job on VSC-4
We have 48 CPUs per Node. In order to fill:
- `--partition=mem_xxxx` (per email)
- `--qos=xxxxxx` (see below)
- `--account=xxxxxx` (see below)
The core hours will be charged to the specified account. If not specified, the default account will be used.
on VSC-4 our account is called:
```
sacctmgr show user `id -u` withassoc format=user,defaultaccount,account,qos%40s,defaultqos%20s
User Def Acct Account QOS Def QOS
---------- ---------- ---------- ---------------------------------------- --------------------
74101 p71386 p71386 p71386_0384 p71386_0384
```
Put this in the Job file:
``` ```
module load intel #!/bin/bash
#
#SBATCH -J TEST_JOB
#SBATCH -N 2
#SBATCH --ntasks-per-node=48
#SBATCH --ntasks-per-core=1
#SBATCH --mail-type=BEGIN # first have to state the type of event to occur
#SBATCH --mail-user=<email@address.at> # and then your email address
#SBATCH --partition=mem_xxxx
#SBATCH --qos=p71386_0384
#SBATCH --account=p71386
#SBATCH --time=<time>
# when srun is used, you need to set (Different from Jet):
<srun -l -N2 -n32 a.out >
# or
<mpirun -np 32 a.out>
``` ```
submit the job
```
[username@l31 ~]$ sbatch check.slrm # to submit the job
[username@l31 ~]$ squeue -u `whoami` # to check the status of own jobs
[username@l31 ~]$ scancel JOBID # for premature removal, where JOBID
# is obtained from the previous command
```
## Software
The VSC use the same software system as Jet and have environmental modules available to the user:
- [VSC Wiki Software](https://wiki.vsc.ac.at/doku.php?id=doku:software)
- VSC-3 has an `anaconda3` module
- VSC-4 has `miniconda3` modules for GNU and INTEL ;)
```
[username@l34 ~]$ module avail # lists the **available** Application-Software,
# Compilers, Parallel-Environment, and Libraries
[username@l34 ~]$ module list # shows currently loaded package of your session
[username@l34 ~]$ module unload <xyz> # unload a particular package <xyz> from your session
[username@l34 ~]$ module load <xyz> # load a particular package <xyz> into your session
```
will load the intel compiler suite and add variables to your environment. will load the intel compiler suite and add variables to your environment.
**Please do not forget to add the module load statements to your jobs.** **Please do not forget to add the module load statements to your jobs.**
0% Loading or .
You are about to add 0 people to the discussion. Proceed with caution.
Please register or to comment