Skip to content
Snippets Groups Projects
Commit 94aa8897 authored by Michael Blaschek's avatar Michael Blaschek :bicyclist:
Browse files

fixed a few VSC-5 things

parent dcee9cab
No related branches found
No related tags found
No related merge requests found
......@@ -67,6 +67,21 @@ $ sqos
the HOME and DATA partition are the same as on [VSC-4](#storage-on-vsc-4).
![JET and VSC-5 holding hands](./mkdocs/img/jet_and_vsc5.png)
since Fall 2023 there has been a major update. JET and VSC-5 are holding hands now. Your files on JET are now accessible from VSC-5. e.g.
```txt title="JET and VSC-5"
a directory on JET
/jetfs/home/[username]
can be found on VSC-5
/gpfs/jetfs/home/[username]
```
You can use these directories as well for direct writing. The performance is higher on VSC-5 storage. This does not work on VSC-4.
## Node Information VSC-4
```txt title='VSC-4 Compute Node'
......@@ -115,6 +130,7 @@ home FILESET 62.17G 100G 100G 207.8M none | 631
## Other Storage
We have access to the Earth Observation Data Center [EODC](https://eodc.eu/data/), where one can find primarily the following data sets:
- Sentinel-1, 2, 3
- Wegener Center GPS RO
......@@ -172,9 +188,10 @@ $ sacctmgr show qos format=name%20s,priority,grpnodes,maxwall,description%40s
SLURM allows for setting a run time limit below the default QOS's run time limit. After the specified time is elapsed, the job is killed:
```bash
```bash title="slurm time limit"
#SBATCH --time=<time>
```
Acceptable time formats include `minutes`, `minutes:seconds`, `hours:minutes:seconds`, `days-hours`, `days-hours:minutes` and `days-hours:minutes:seconds`.
......@@ -194,7 +211,7 @@ The core hours will be charged to the specified account. If not specified, the d
Put this in the Job file (e.g. VSC-5 Nodes)
```bash
```bash title="VSC slurm example job"
#!/bin/bash
#
#SBATCH -J TEST_JOB
......@@ -224,7 +241,7 @@ Put this in the Job file (e.g. VSC-5 Nodes)
* **--mail-type** sends an email at specific events. The SLURM doku lists the following valid mail-type values: *"BEGIN, END, FAIL, REQUEUE, ALL (equivalent to BEGIN, END, FAIL and REQUEUE), TIME_LIMIT, TIME_LIMIT_90 (reached 90 percent of time limit), TIME_LIMIT_80 (reached 80 percent of time limit), and TIME_LIMIT_50 (reached 50 percent of time limit). Multiple type values may be specified in a comma separated list."* [cited from the SLURM doku](http://slurm.schedmd.com)
* **--mail-user** sends an email to this address
```bash
```bash title="slurm basic commands"
sbatch check.slrm # to submit the job
squeue -u `whoami` # to check the status of own jobs
scancel JOBID # for premature removal, where JOBID
......@@ -238,7 +255,7 @@ The VSC use the same software system as Jet and have environmental modules avail
- [VSC Wiki Software](https://wiki.vsc.ac.at/doku.php?id=doku:software)
- VSC-4 has `miniconda3` modules for GNU and INTEL ;)
```bash
```bash title="VSC modules"
module avail # lists the **available** Application-Software,
# Compilers, Parallel-Environment, and Libraries
module list # shows currently loaded package of your session
......@@ -252,13 +269,17 @@ will load the intel compiler suite and add variables to your environment.
on how to use environment modules go to [Using Environment Modules](Misc/Environment-Modules.md)
### Import user-site packages
It is possible to install user site packages into your `.local/lib/python3.*` directory:
```bash
```bash title="installing python packages in your HOME"
# installing a user site package
/gpfs/data/fs71386/imgw/run.sh pip install --user [package]
pip install --user [package]
```
```python
**Please remember that all HOME and DATA quotas will be shared** Installing a lot of packages creates a lot of files!
```python title="Python importing user site packages"
import sys, site
sys.path.append(site.site.getusersitepackages())
# This will add the correct path.
......
mkdocs/img/jet_and_vsc5.png

6.98 KiB

0% Loading or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment