diff --git a/VSC.md b/VSC.md index 1ea99165692b327fe459f3cd98e67c077d3098bc..47996d1e6acc57133702f60abc38ce2d5485b7ff 100644 --- a/VSC.md +++ b/VSC.md @@ -67,6 +67,21 @@ $ sqos the HOME and DATA partition are the same as on [VSC-4](#storage-on-vsc-4). + + +since Fall 2023 there has been a major update. JET and VSC-5 are holding hands now. Your files on JET are now accessible from VSC-5. e.g. + +```txt title="JET and VSC-5" +a directory on JET +/jetfs/home/[username] + +can be found on VSC-5 +/gpfs/jetfs/home/[username] + +``` + +You can use these directories as well for direct writing. The performance is higher on VSC-5 storage. This does not work on VSC-4. + ## Node Information VSC-4 ```txt title='VSC-4 Compute Node' @@ -115,6 +130,7 @@ home FILESET 62.17G 100G 100G 207.8M none | 631 ## Other Storage We have access to the Earth Observation Data Center [EODC](https://eodc.eu/data/), where one can find primarily the following data sets: + - Sentinel-1, 2, 3 - Wegener Center GPS RO @@ -172,9 +188,10 @@ $ sacctmgr show qos format=name%20s,priority,grpnodes,maxwall,description%40s SLURM allows for setting a run time limit below the default QOS's run time limit. After the specified time is elapsed, the job is killed: -```bash +```bash title="slurm time limit" #SBATCH --time=<time> ``` + Acceptable time formats include `minutes`, `minutes:seconds`, `hours:minutes:seconds`, `days-hours`, `days-hours:minutes` and `days-hours:minutes:seconds`. @@ -194,7 +211,7 @@ The core hours will be charged to the specified account. If not specified, the d Put this in the Job file (e.g. VSC-5 Nodes) -```bash +```bash title="VSC slurm example job" #!/bin/bash # #SBATCH -J TEST_JOB @@ -224,7 +241,7 @@ Put this in the Job file (e.g. VSC-5 Nodes) * **--mail-type** sends an email at specific events. The SLURM doku lists the following valid mail-type values: *"BEGIN, END, FAIL, REQUEUE, ALL (equivalent to BEGIN, END, FAIL and REQUEUE), TIME_LIMIT, TIME_LIMIT_90 (reached 90 percent of time limit), TIME_LIMIT_80 (reached 80 percent of time limit), and TIME_LIMIT_50 (reached 50 percent of time limit). Multiple type values may be specified in a comma separated list."* [cited from the SLURM doku](http://slurm.schedmd.com) * **--mail-user** sends an email to this address -```bash +```bash title="slurm basic commands" sbatch check.slrm # to submit the job squeue -u `whoami` # to check the status of own jobs scancel JOBID # for premature removal, where JOBID @@ -238,7 +255,7 @@ The VSC use the same software system as Jet and have environmental modules avail - [VSC Wiki Software](https://wiki.vsc.ac.at/doku.php?id=doku:software) - VSC-4 has `miniconda3` modules for GNU and INTEL ;) -```bash +```bash title="VSC modules" module avail # lists the **available** Application-Software, # Compilers, Parallel-Environment, and Libraries module list # shows currently loaded package of your session @@ -252,13 +269,17 @@ will load the intel compiler suite and add variables to your environment. on how to use environment modules go to [Using Environment Modules](Misc/Environment-Modules.md) ### Import user-site packages + It is possible to install user site packages into your `.local/lib/python3.*` directory: -```bash + +```bash title="installing python packages in your HOME" # installing a user site package -/gpfs/data/fs71386/imgw/run.sh pip install --user [package] +pip install --user [package] ``` -```python +**Please remember that all HOME and DATA quotas will be shared** Installing a lot of packages creates a lot of files! + +```python title="Python importing user site packages" import sys, site sys.path.append(site.site.getusersitepackages()) # This will add the correct path. diff --git a/mkdocs/img/jet_and_vsc5.png b/mkdocs/img/jet_and_vsc5.png new file mode 100644 index 0000000000000000000000000000000000000000..a6aa149b9f241d5a4f86d26aed101d1cab014460 Binary files /dev/null and b/mkdocs/img/jet_and_vsc5.png differ