diff --git a/Documentation/GPFS-jet.png b/Documentation/GPFS-jet.png
deleted file mode 100644
index c59757d67764269ec1b974d55a00a75ad0aef4f1..0000000000000000000000000000000000000000
Binary files a/Documentation/GPFS-jet.png and /dev/null differ
diff --git a/Jet-Cluster.md b/Jet-Cluster.md
index 25bb1c6429b0d387551ffeae48d4da3e320696c2..0942dea61bedca602285d21427de8cc110da2ffa 100644
--- a/Jet-Cluster.md
+++ b/Jet-Cluster.md
@@ -1,7 +1,7 @@
-# The Jet Cluster
-[@IMG University of Vienna](https://jet01.img.univie.ac.at) :rocket:
+# The J E T Cluster
-[[_TOC_]]
+> Research Cluster for Staff
+> :rocket: 🔒
## Getting Started
@@ -9,6 +9,7 @@ Welcome to the HPC @IMG @UNIVIE and please follow these steps to become a produc
**Efficiency is keen.**
Steps:
+
1. Request access
2. Connect to Jet
- [How to SSH / VNC / VPN](SSH-VPN-VNC/README.md)
@@ -19,12 +20,15 @@ Steps:
## System Information
+
Last Update: 4.12.2020
-Node Setup
+
+Node Setup:
+
- 2x Login Nodes
- 7x Compute Nodes
-
+
| Name | Value |
@@ -37,12 +41,14 @@ Node Setup
| Memory/Core | 18.9 GB |
| Network | 40 Gbit/s (Infiniband) |
-Global file system (GPFS) is present on all nodes with about 1 PB (~1000 TB) of storage.
+Global file system (GPFS) is present on all nodes with about 1.5 PB (~1500 TB) of storage.
## Software
The typcial installation of a intel-cluster has the INTEL Compiler suite (`intel-parallel-studio`) and the open source GNU Compilers installed. Based on these two different compilers (`intel`, `gnu`), there are usually two version of each scientific software.
+
Major Libraries:
+
- OpenMPI (3.1.6, 4.0.5)
- HDF5
- NetCDF (C, Fortran)
@@ -53,7 +59,7 @@ Major Libraries:
These software libraries are usually handled by environment modules.
-
+
## Currently installed modules
@@ -93,36 +99,41 @@ on how to use environment modules go to [Using Environment Modules](Misc/Environ
## Jupyterhub
-<img src="Documentation/jupyterhub-logo.svg" width="100px">
+<img src="/mkdocs/img/jupyterhub-logo.svg" width="100px">
-The Jet-cluster serves a [jupyterhub](https://jupyterhub.readthedocs.io/en/stable/) with a [jupyterlab](https://jupyterlab.readthedocs.io/en/stable/) that launches on the jet-cluster compute nodes and allows users to work directly on the cluster as well as submit jobs.
+The Jet Cluster serves a [jupyterhub](https://jupyterhub.readthedocs.io/en/stable/) with a [jupyterlab](https://jupyterlab.readthedocs.io/en/stable/) that launches on the jet-cluster compute nodes and allows users to work directly on the cluster as well as submit jobs.
-Goto: [](https://jet01.img.univie.ac.at) from within the VPN or UNI-Network.
-Login with your jet-Credentials, choose a job and the jupyterlab will be launched.
+Steps:
-<img src="Documentation/jet-login.png" width="500px">
+ - [https://jet01.img.univie.ac.at](https://jet01.img.univie.ac.at) from within the VPN or UNI-Network.
+ - Login with your Jet Credentials
+ - Choose a job
+ - The jupyterlab will be launched and will be available to you until you log out or the wall time exceeds.
-<img src="Documentation/jet-job1.png" width="500px">
+<img src="/mkdocs/img/jet-login.png" width="500px">
-<img src="Documentation/jet-job2.png" width="500px">
+<img src="/mkdocs/img/jet-job1.png" width="500px">
+
+<img src="/mkdocs/img/jet-job2.png" width="500px">
There are several kernels available and some help can be found:
- - [Python/](Python/)
+
- [Tutorial on Jet](Python/Your-First-Notebook-onJet_v2.ipynb)
## User Quotas and Restrictions
Please try to use the compute nodes in a responsible manner. Currently there are not restrictions on the duration or the resources you can request. However, please follow these rules of collaboration:
-Jobs
-- CPU, keyword: `ntasks` e.g. 1 Node == 2x20 physcial cores
+Jobs:
+
+- Number of CPUs, keyword: `ntasks` e.g. 1 Node == 2x20 physcial cores
- Memory, keyword: `mem` e.g. each Node up to 754 GB
- Runtime, keyword: `time` e.g. try to split jobs into pieces.
Consider the following example. You can use one node relatively easy for more than 3 days with your jobs running, but do not use all nodes an block them for all other users for 3 days. If you need multiple nodes, split the jobs into shorter runtimes. In general it is better to have more smaller jobs that are processed in a chain. Also try not to use too much resources that get wasted. Have a look at resources used in your jobs using the `/usr/bin/time` command or look [here](https://gitlab.phaidra.org/imgw/slurm).
Sample Job
-```
+```bash
#!/bin/bash
# SLURM specific commands
#SBATCH --job-name=test-run
@@ -148,7 +159,7 @@ On the Login Nodes (jet01/jet02) jobs can only use 20 GB of Memory as the rest n
## Network drives
-Currently there are two network drives mounted on the jet cluster. These are connected to the [SRVX8](SRVX8.md) and data can be transfered or accessed like this on the jet cluster. Be aware that the data needs to be transfered via the network before and latencies are higher.
+Currently there are two network drives mounted on the jet cluster. These are connected to the [SRVX1](SRVX1.md) and data can be transfered or accessed like this on the jet cluster. Be aware that the data needs to be transfered via the network before and latencies are higher.
```
131.130.157.8:/mnt/scratch 400T 378T 22T 95% /mnt/scratch
131.130.157.8:/mnt/users 319T 309T 11T 97% /mnt/users
@@ -156,4 +167,5 @@ jetfs 1.1P 81T 975T 8% /jetfs
```
## Slurm
+More advanced slurm tutorial and recommended usage.
[Slurm Tutorial on Gitlab](https://gitlab.phaidra.org/imgw/slurm) 🔒
diff --git a/Python/bufr2synop.py b/Python/bufr2synop.py
new file mode 100644
index 0000000000000000000000000000000000000000..7c955814831bf0c30e70cd700da2f0371220068c
--- /dev/null
+++ b/Python/bufr2synop.py
@@ -0,0 +1,15 @@
+
+import eccodes
+import pdbufr
+
+ keys = ['blockNumber',
+ 'stationNumber',
+'latitude',
+'longitude',
+'airTemperature',
+'dewpointTemperature',
+ 'windSpeed',
+'windDirection',
+]
+
+df = pdbufr.read_bufr('ISMD22_LOWM.010000', columns=keys)
\ No newline at end of file
diff --git a/README.md b/README.md
index 78a766851bcb12bf26d3482ff13a7f3d06441f39..97e74c8e7c19d1c4628983c78ae4355a53cc178e 100644
--- a/README.md
+++ b/README.md
@@ -1,6 +1,6 @@
-<img src="https://homepage.univie.ac.at/michael.blaschek/img/logo_fgga.jpg" width="300px">
-<img src="https://homepage.univie.ac.at/michael.blaschek/img/logo_img.png" width="300px">
-<img src="https://homepage.univie.ac.at/michael.blaschek/img/rocket.png" width="100px">
+<img src="./mkdocs/img/logo_uniwien.jpg" width="300px">
+<img src="./mkdocs/img/logo_img2_color.png" width="300px">
+<img src="./mkdocs/img/rocket.png" width="100px">
# Welcome to the Department of Meteorology and Geophysics
[@University of Vienna](https://univie.ac.at)
@@ -19,10 +19,14 @@ If you care to participate please do so. Raise an Issue or give some feedback.
Missing solutions are very welcome.
-# Available Servers
+# Available Work Environments
-- [SRVX1](SRVX1.md), [SRVX8](SRVX8.md) @ UZA2
-- [Jet Cluster @ Arsenal](Jet-Cluser.md)
-- [Vienna Scientific Cluster (VSC)](VSC.md)
+Locations:
+
+- Staff + Students [SRVX1](SRVX1.md)
+- Staff + Remote Desktop [SRVX8](SRVX8.md)
+- Staff + Remote Desktop + Jupyterhub [Jet Cluster](Jet-Cluser.md)
+- Staff + Students, Jupyterhub called [TeachingHub](TeachingHub.md)
+- Staff [Vienna Scientific Cluster (VSC)](VSC.md)
- [VSC Training](https://vsc.ac.at/training)
- [VSC Trainings @ IMGW](https://gitlab.phaidra.org/imgw/trainings-course)
diff --git a/SRVX1.md b/SRVX1.md
index f8f0d5758b783994275cd6b76e523c7110817aec..daf0c24db23890daae90afa1319f56633d6c101d 100644
--- a/SRVX1.md
+++ b/SRVX1.md
@@ -1,20 +1,17 @@
# S R V X 1
-[[_TOC_]]
+> Teaching Hub and Access Point from the web
+>
## Getting Started
Steps:
-1. Request access
-2. Access using SSH - [How to SSH / VNC / VPN](SSH-VPN-VNC/README.md)
-3. Access using Teaching Hub - [How to connect using the TeachingHub](TeachingHub.md)
-4. Access Services:
- - [Webdata](https://srvx1.img.univie.ac.at/webdata) - Hosting files
- - [Yopass](https://srvx1.img.univie.ac.at/secure) - Encrypted message service
- - [ECGateway](https://srvx1.img.univie.ac.at/ecmwf/ecmwf) (UNIVIE Network, [VPN](SSH-VPN-VNC/VPN.md))
- - [Transfer.sh](https://srvx1.img.univie.ac.at/filetransfer) ()
- - [iLibrarian](https://srvx1.img.univie.ac.at/)
+
+1. Request access / will be done for you by the lecturer
+2. As Staff, access using SSH - [How to SSH / VNC / VPN](SSH-VPN-VNC/README.md)
+3. As Student, access using Teaching Hub - [How to connect using the TeachingHub](TeachingHub.md)
+4. Access Services [https://srvx1.img.univie.ac.at](https://srvx1.img.univie.ac.at)
## System Information
| Name | Value |
@@ -26,6 +23,7 @@ Steps:
| Memory | 754 GB Total |
| Memory/Core | 9.4 GB |
+### Greeter
```
----------------------------------------------
131.130.157.11 _ . , . .
@@ -42,26 +40,27 @@ Steps:
## Services
The SRVX1 is the central access point to IMG services:
goto: [srvx.img.univie.ac.at](https://srvx1.img.univie.ac.at)
+
Currently running:
+
- TeachingHub (Jupyterhub)
- Webdata - File hosting
- YoPass - Password Sharing
- iLibrarian - Paper Management (development)
- Filetransfer.sh - File sharing service (development)
-## Jupyterhub
-<img src="Documentation/jupyterhub-logo.svg" width="300px">
-SRVX1 serves a teaching [jupyterhub](https://jupyterhub.readthedocs.io/en/stable/) with a [jupyterlab](https://jupyterlab.readthedocs.io/en/stable/). It allows easy access for students and teachers.
+## Jupyterhub
+<img src="/mkdocs/img/jupyterhub-logo.svg" width="300px">
-Goto: [https://srvx1.img.univie.ac.at/hub](https://srvx1.img.univie.ac.at/hub)
+SRVX1 serves a teaching [jupyterhub](https://jupyterhub.readthedocs.io/en/stable/) with a [jupyterlab](https://jupyterlab.readthedocs.io/en/stable/). It allows easy access for students and teachers. Access: [https://srvx1.img.univie.ac.at/hub](https://srvx1.img.univie.ac.at/hub)
Signup is only granted by teachers and requires a srvx1 user account. A new password is needed and a TOTP (time base one-time password) will be created.
Download/Use any of these required Authenticator Apps:
+
- [2FAS (Mobile, recommended)](https://2fas.com/)
- [KeepassX (Desktop)](https://www.keepassx.org/)
- - [Authy (Mobile, Desktop)](https://authy.com/download/)
- [FreeOTP (Mobile)](https://freeotp.github.io/)
- [Google Auth (Mobile)](https://play.google.com/store/apps/details?id=com.google.android.apps.authenticator2)
@@ -71,7 +70,9 @@ After registering the teacher/admin has to grant you access and you can login.
## Software
The typcial installation of a intel-server has the INTEL Compiler suite (`intel-parallel-studio`, `intel-oneapi`) and the open source GNU Compilers installed. Based on these two different compilers (`intel`, `gnu`), there are usually two version of each scientific software.
+
Major Libraries:
+
- OpenMPI (3.1.6, 4.0.5)
- HDF5
- NetCDF (C, Fortran)
@@ -82,7 +83,7 @@ Major Libraries:
These software libraries are usually handled by environment modules.
-
+
## Currently installed modules
Please note that new versions might already be installed.
@@ -173,4 +174,17 @@ containers:
- rtcoef
- rthelp
- rttest
+ - description: Use for running RTTOV simulations with Python.
+ - LIBEMOS:
+ - LIBEMOS: 4.5.1
+ - compiler: gcc:7.5.0
+ - path: /home/swd/containers/libemos-dev/libemos-dev.sif
+ - os: ubuntu:18.04
+ - singularity: 3.8.6
+ - packages:
+ - openmpi:2.1.1-8
+ - fftw3:3.3.7-1
+ - eccodes:2.6.0
+ - libemos
+ - description: Use for building flexextract with working libemos dependencies.
```
diff --git a/SRVX8.md b/SRVX8.md
index 1108107331bf4e3affa88573404877b8f3661fb6..0e4b0ebcbd429dd979e657ab5626ecdc88c2f017 100644
--- a/SRVX8.md
+++ b/SRVX8.md
@@ -1,11 +1,13 @@
# S R V X 8
-[[_TOC_]]
+> Remote Desktop and Virtual Machines
+>
## Getting Started
Steps:
+
1. Request access
2. Access using SSH - [How to SSH / VNC / VPN](SSH-VPN-VNC/README.md)
@@ -20,6 +22,7 @@ Steps:
| Memory | 504 GB Total |
| Memory/Core | 18 Gb |
+### Greeter
```
----------------------------------------------
_
@@ -41,7 +44,9 @@ Steps:
## Software
The typcial installation of a intel-server has the INTEL Compiler suite (`intel-parallel-studio`, `intel-oneapi`) and the open source GNU Compilers installed. Based on these two different compilers (`intel`, `gnu`), there are usually two version of each scientific software.
+
Major Libraries:
+
- OpenMPI (3.1.6, 4.0.5)
- HDF5
- NetCDF (C, Fortran)
@@ -52,7 +57,7 @@ Major Libraries:
These software libraries are usually handled by environment modules.
-
+
## Currently installed modules
@@ -119,41 +124,18 @@ These scripts are intended to help with certain known problems.
Report problems to: michael.blaschek@univie.ac.at
```
-These are scripts in a common directory. Feel free to copy or edit as you like. Note that some services like filesender require an ACONET account (accessible via your u:account). Please note the available VNC services.
+These are scripts in a common directory. Feel free to copy or edit as you like. Note that some services like filesender require an **ACONET account** (accessible via your u:account). **Please note the available VNC services** for the remote desktop environment.
+For VNC the user can choose between installed desktops:
+
+- xfce
+- icewm (default)
## Virtual Machine Hub
Currently the system acts as a virtual machine host.
+
Active:
+
- VERA
- Ubuntu Geographic
-
-
-## Container Hub
-
-Currently there is the possibility to run [singularity](https://singularity.hpcng.org/) containers on all our Servers. This is really similar to docker, but much more secure for multi-user servers. Almost every docker container can be converted into a singularity container. Some of the build recipes use docker.
-
-There are a number of prepared containers but more can be added. If you have a wish or an existing container useful for others please share.
-```yaml
-containers:
- - root: /home/swd/containers
- - available:
- - RTTOV:
- - RTTOV: 12.3
- - compiler: gcc:7.3.0 (anaconda)
- - path: /home/swd/containers/rttov-jupyter/jup3rttov.sif
- - os: centos:6.10
- - python: 3.7.4
- - singularity: 3.5.2
- - packages:
- - anaconda3
- - jupyter jupyterlab numpy matplotlib pandas xarray bottleneck dask numba scipy netcdf4 cartopy h5netcdf nc-time-axis cfgrib eccodes nodejs
- - apps:
- - atlas
- - lab
- - notebook
- - rtcoef
- - rthelp
- - rttest
-```
diff --git a/SSH-VPN-VNC/Questions.md b/SSH-VPN-VNC/Questions.md
index 0f84e97f6983b6b148b2fe799fac8e54b933f2b1..e41cce55b5e52729966a11076df880e55838e6dd 100644
--- a/SSH-VPN-VNC/Questions.md
+++ b/SSH-VPN-VNC/Questions.md
@@ -1,9 +1,6 @@
# Questions and Answers
Please feel free to add you questions and anwsers.
-[[_TOC_]]
-
-
## Q: How to use ssh-key authentication?
In order to connect passwordless to a remote server a key needs to be generated.
@@ -110,7 +107,7 @@ $ connect2jet -g login jet
## Q: How to mount a remote file system on Linux (MAC)?
You can us programs like [Filezilla](https://filezilla-project.org/) or [Cyberduck](https://cyberduck.io/) (MAC) to transfer files between remote and local host. But sometimes it is much easier to mount a remote file system and work on it like an external drive.
-You can use the [mountserver](mountserver) script to do so. This requires `sshfs` to be installed, on Linux that is in the standard repositories. A short into and some additional steps can be found [here](https://www.tjansson.dk/2008/01/autofs-and-sshfs-the-perfect-couple/).
+You can use the `mountserver` ([Download: mountserver](mountserver)) script to do so. This requires `sshfs` to be installed, on Linux that is in the standard repositories. A short into and some additional steps can be found [here](https://www.tjansson.dk/2008/01/autofs-and-sshfs-the-perfect-couple/).
```bash
$ mountserver [remote] [remote directory] [local directory]
@@ -120,5 +117,10 @@ $ mountserver srvx1 /raid61/wetter /home/monkey/srvx1
This will mount the remote directory to the local directory. The local directory will be created if it does not exist. The directory should be empty before mounting, otherwise that will cause problems.
+??? note "mountserver"
+
+ ``` bash title="Mount a remote directory"
+ --8<-- "SSH-VPN-VNC/mountserver"
+ ```
diff --git a/SSH-VPN-VNC/README.md b/SSH-VPN-VNC/README.md
index 7b2c5d3f22ea266c5c819dd01ad05291a09addd8..249eefa38df0eba0d339fc2c6bbb8be812c27449 100644
--- a/SSH-VPN-VNC/README.md
+++ b/SSH-VPN-VNC/README.md
@@ -1,24 +1,29 @@
-# SSH, VPN, VNC related Problems
+# Connection 101
-[[_TOC_]]
+Please find here some details on how to connect using SSH or VPN or VNC. And some other linux related terminal based information.
## General Access
Please use these methods to access the servers:
+
1. Connect using [SSH](SSH.md) or [MOSH](https://mosh.org/#) (Mobile Secure Shell)
2. Connect using [TeachingHub](../TeachingHub.md) or [ResearchHub](../Jet-Cluster.md#jupyterhub)
3. Connect using [VNC](VNC.md)
4. Access from outside the UNI, [VPN](VPN.md)
## Jupyterhub
+
Currently, there are two JupyterHub instances runnning:
+
- TeachingHub on SRVX1
- ResearchHub on Jet01
-Connect to either
+Connect to either :
+
- [https://srvx1.img.univie.ac.at](https://srvx1.img.univie.ac.at) from everywhere
- [https://jet01.img.univie.ac.at](https://jet01.img.univie.ac.at) from within the university network.
## Screen
+
[Screen](https://wiki.ubuntuusers.de/Screen/) is terminal session manager, that allows to start processes and reconnect to these processes after disconnection.
This starts a new session
@@ -30,7 +35,7 @@ You can detach from this session with `CTRL + A D` and reconnect again with `scr
Multiple Sessions can be created and the output saved (`-L` Option).
-
+
## Tmux
[Tmux](https://wiki.ubuntuusers.de/tmux/) is a terminal multiplexer, that allows to open more consoles and allows to detach the session. It is much more complex and powerful compared to screen.
@@ -41,6 +46,7 @@ Launches a new virtual terminal, with `CTRL + B D` it can bed detached and with
## Questions and Answers
+
- [Q: How to use ssh-key authentication?](Questions.md#q-how-to-use-ssh-key-authentication)
- [Q: How to use an ssh-agent?](Questions.md#q-how-to-use-an-ssh-agent)
- [Q: How to transfer files between two VPN networks?](Questions.md#q-how-to-transfer-files-between-two-vpn-networks)
@@ -48,8 +54,28 @@ Launches a new virtual terminal, with `CTRL + B D` it can bed detached and with
- [Q: How to mount a remote file system on Linux (MAC)?](Questions.md#q-how-to-mount-a-remote-file-system-on-Linux-mac)
## Tools
-Please find some useful tools for connecting to IMGW servers and University of Vienna VPN.
-- BASH script using SSH to connect via a gateway, [SSH](SSH.md#connect-script) [connect2jet](connect2jet)
-- BASH script for 5fpc tools, [VPN](VPN.md#connect-script) [connect2vpn](connect2vpn)
-- Change VNC resolution, [VNC](VNC.md#xrandr) [add_xrandr_resolution](add_xrandr_resolution.sh)
+
+Please find some useful tools for connecting to IMGW servers and University of Vienna VPN:
+
+- BASH script using SSH to connect via a gateway, [SSH](SSH.md#connect-script) Download: [connect2jet](connect2jet)
+- BASH script for 5fpc tools, [VPN](VPN.md#connect-script) Download: [connect2vpn](connect2vpn)
+- Change VNC resolution, [VNC](VNC.md#xrandr) Download: [add_xrandr_resolution](add_xrandr_resolution.sh)
- Mount Server directories via sshfs, [SSHFS](SSH.md#sshfs)
+
+??? note "connect2jet"
+
+ ``` bash title="Connect to Jet"
+ --8<-- "SSH-VPN-VNC/connect2jet"
+ ```
+
+??? note "connect2vpn"
+
+ ``` bash title="Connect to VPN"
+ --8<-- "SSH-VPN-VNC/connect2vpn"
+ ```
+
+??? note "add_xrandr_resolution"
+
+ ``` bash title="Add a new monitor resolution to VNC"
+ --8<-- "SSH-VPN-VNC/add_xrandr_resolution.sh"
+ ```
\ No newline at end of file
diff --git a/SSH-VPN-VNC/SSH.md b/SSH-VPN-VNC/SSH.md
index 80aa79532df0070afa14e76b60946551d40790be..ce5a515f4d838f6ce84f75961b57b8d25cef3755 100644
--- a/SSH-VPN-VNC/SSH.md
+++ b/SSH-VPN-VNC/SSH.md
@@ -66,11 +66,17 @@ If you are a guest, you can apply for a [guest u:account](https://zid.univie.ac.
Find a solution [Questions - How to use ssh-key authentication?](Questions.md#q-how-to-use-ssh-key-authentication) or [Questions - How to use an ssh-agent?](Questions.md#q-how-to-use-an-ssh-agent)
## Connect Script
-If you are using a terminal (Mac, Linux, WSL, ...) you can use the script [connect2jet](connect2jet) like this:
+If you are using a terminal (Mac, Linux, WSL, ...) you can use the script [Download: connect2jet](connect2jet) like this:
```bash
connect2jet -g [U:Account-Username]@login.univie.ac.at [Jet-Username]@jet01.img.univie.ac.at
```
+??? note "connect2jet"
+
+ ``` bash title="Connect to Jet"
+ --8<-- "SSH-VPN-VNC/connect2jet"
+ ```
+
There is also an option to forward a port, e.g. the VNC Port:
```bash
diff --git a/SSH-VPN-VNC/VNC.md b/SSH-VPN-VNC/VNC.md
index cdbcb901912e7e0a50eaf985208c4787a2be0964..3c56eb31bb9d154505655bc8175e139905d2c547 100644
--- a/SSH-VPN-VNC/VNC.md
+++ b/SSH-VPN-VNC/VNC.md
@@ -45,6 +45,7 @@ Running the script without any options will run all necessary steps. In case of
Please consider using the `userservices vnc` script to do this setup.
First of all check if a VNC server is already running or not. Depending on the results you have two options:
+
1. Use an existing. (Note the Port/Display Number)
2. Stop all and start a new VNC server
3. **on Jet** setup a service running your VNC Server.
@@ -78,6 +79,7 @@ Use a VNC client (e.g. [TigerVNC Viewer](https://sourceforge.net/projects/tigerv
**Note: Since version 1.10 of tigerVNC it is possible to resize the windows directly**
Connect with a viewer:
+
1. Hostname: `srvx1.img.univie.ac.at` or `jet01.img.univie.ac.at`
2. VNC Port: `vncserver -list` or `systemctl status --user vncserver.slice`
3. VNC passwort (`vncpasswd` allows to set this on the server)
diff --git a/SSH-VPN-VNC/VPN.md b/SSH-VPN-VNC/VPN.md
index 226187173e6183aad6ec2411a0c010561c61aa07..4e2d85ee7a7ca3e5f6b693a955ffabc51f9c5e5b 100644
--- a/SSH-VPN-VNC/VPN.md
+++ b/SSH-VPN-VNC/VPN.md
@@ -1,5 +1,7 @@
# VPN
+
Requirements:
+
- `u:account`
Some servers are only accessible from within the virtual network of the university of Vienna. Therefore access from outside has to be granted via the [VPN-Service](https://vpn.univie.ac.at). Go there and login with your `u:account` and download the *Big-IP Edge* client for you system.
@@ -7,6 +9,7 @@ Some servers are only accessible from within the virtual network of the universi
Links:
+
* [ZID-VPN](https://vpn.univie.ac.at/f5-w-68747470733a2f2f7a69642e756e697669652e61632e6174$$/vpn/)
* Linux (Ubuntu, Generic), Windows, Mac: [VPN user guides](https://vpn.univie.ac.at/f5-w-68747470733a2f2f7a69642e756e697669652e61632e6174$$/vpn/anleitungen/)
* Arch based AUR package [AUR f5fpc](https://aur.archlinux.org/packages/f5fpc/)
diff --git a/TeachingHub.md b/TeachingHub.md
index 4827d45bbec1af3c137323a3f1a75d268c4fe743..2698e1cc63e75a345eda06f9b2ab37830238a852 100644
--- a/TeachingHub.md
+++ b/TeachingHub.md
@@ -1,74 +1,17 @@
-# Teaching Hub @ IMGW
+# Teaching Hub
-<img src="Documentation/jupyterhub-logo.svg" width="300px">
+<img src="/mkdocs/img/jupyterhub-logo.svg" width="100px">
-- Address: [Teaching Hub](https://srvx1.img.univie.ac.at)
-- Access to the Teaching Hub can be done only via a browser.
-- Access to SRVX1 via a terminal (Putty,...)
+## Getting Started
+Address: [https://srvx1.img.univie.ac.at/hub](https://srvx1.img.univie.ac.at/hub)
-
-[[_TOC_]]
+Access to the Teaching Hub in a browser (mobile or desktop)
## How to get access?
-Usually **your teacher** is going to **sign you up** and you will recieve one or two emails containing information on how to login.
-You will recieve access to the Teaching Hub usually for 6 month. Can be extended for good reasons or until a teacher signs you up again.
-
-## You recieved two Emails
-This means you are new to the SERVERS and the Teaching Hub.
-
-The first email contains your login details for SRVX1, you get a username (Matrikelnumber) and a password. It looks like this:
-```
-Dear TeachingHub User Monkey
-Some of your user information has been updated.
-
-Server Account Specification:
-servers: srvx1.img.univie.ac.at
-username: monkey
-password: ABCDEFGHIJK (Change as soon as possible on all servers)
-uid: 40XXX
-gid: 59000
-Name: Monkey Banana
-Email: a000000@unet.univie.ac.at
-GECOS: Monkey Banana (a000000@unet.univie.ac.at)
-home: /home/srvx11/lehre/users/a000000
-shell: /bin/bash
-Account expires: 2021-08-30
-...
-```
-Use these to login to the server and change the password to something better and still something that is safe:
-```bash
-$ ssh [user]@srvx1.img.univie.ac.at
-----------------------------------------------
- _
- (` ).
- ( ).
-) _( SRVX1 '`.
- .=(`( . ) .--
- (( (..__.:'-' .+( )
-`. `( ) ) ( . )
- ) ` __.:' ) ( ( ))
-) ) ( ) --' `- __.'
-.-' (_.' .')
- (_ )
- 131.130.157.5
---..,___.--,--'`,---..-.--+--.,,-,,.-..-._.-.-
-----------------------------------------------
-Teaching Server @IMGW
-Environmental Modules: module av
-Jupyterhub: https://srvx1.img.univie.ac.at
-Usage Profile: Students
-----------------------------------------------
-
-$ passwd
-Changing password for user [user].
-Changing password for [user].
-(current) UNIX password:
-```
-After you completed these steps you are free to login anytime and use the server.
-The second email contains your Teaching Hub credentials. See below.
+Usually **your teacher** is going to **sign you up** and you will receive one or two emails containing information on how to login.
+You will receive access to the Teaching Hub usually for 6 month. Can be extended for good reasons or until a teacher signs you up again.
-## You recieved one Email
-the email you recieved contains credentials for the Teaching Hub. This email contains your username (the same as on SRVX1 for login), a new password (only for the Teaching Hub) and a secret (16 digit code). They look like this:
+The email you received contains credentials for the Teaching Hub. This email contains your username, a new password (only for the Teaching Hub) and a secret (16 digit code). They look like this:
```
Dear TeachingHub User Monkey
Some of your user information has been updated.
@@ -76,61 +19,61 @@ Some of your user information has been updated.
TeachingHub Account Specification:
servers: https://srvx1.img.univie.ac.at
username: monkey
-password: ABCDEFGHIJK (signed in, click on user name, upper right corner)
+password: ABCDEFGHIJK (signed in, click on user name, upper right corner to change)
Email: a000000@univie.ac.at
Secret: 1A2B3C4D5E6F7G8H
Please note that the TeachingHub is open from the web.
-Two-factor Authentication is required. Download one of these apps and use your secret to generate TOTP (Time based One Time Passwords).
-Authy (https://authy.com/download/)
-FreeOTP (https://freeotp.github.io/)
-Google Auth (https://play.google.com/store/apps/details?id=com.google.android.apps.authenticator2)
+Two-factor Authentication is required. Download any TOTP (Time based One Time Passwords) app according to your OS.
...
```
-You need to download one of these apps:
+
+As stated in the email you received, you need to download one of these TOTP apps:
+
- [2FA Authenticator](https://2fas.com) :rocket: (android, ios)
-- [Authy](https://authy.com/download/)
- [FreeOTP](https://freeotp.github.io/)
- [Google Auth](https://play.google.com/store/apps/details?id=com.google.android.apps.authenticator2)
- or any other (search `2fa apps` and you will find plenty)
I recommend using your phone and an app there, as this is the most secure way. And with you phone you can scan the qr code, which looks like this:
-<img src="Documentation/teachinghub-qrcode.png" width="300px">
+<img src="/mkdocs/img/teachinghub-qrcode.png" width="300px">
When you scan to your Authenticator App, it will automatically get the details (username, Secret) required to generate every 30 seconds a Time-based One Time Password (TOTP). Like this:
-
+
If you want to use a desktop or other app that might not scan QR codes, you can fill in the details manually like this:
- Provider: `TeachingHub`
- Account: `[user]` your username, here `monkey`
- Token or Secret or Hash: `1A2B3C4D5E6F7G8H` is the secret from your email.
-
+
The combination of username, password and TOTP needs to be used to sign in:
-
+
This sounds all really complicated, but I promise this will be easier from time to time. Don't forget that this for our all security and your contribution is appreciated! Thanks.
+If you have trouble with login, please check if you device has time synchronization activated, otherwise you might just get wrong codes. (Nice for checking: [totp.app](https://totp.app) *However, be warned this is an online service and your data might not be as secure as you think!*)
+
Now you are ready to go to the teaching hub and sign in:
[Teaching Hub](https://srvx1.img.univie.ac.at)
-# Teaching Hub Notebooks and Kernels
+## Teaching Hub Notebooks and Kernels
Currently there are several Notebooks with different Kernels (Python kernels) installed at SRVX1. Look at the logos, if SRVX1 is in there then you will have access to these notebooks and kernels. If SRVX2 or SRVX8 are in the logo you might not have access as this is for researchers.
-
+
As you can see it is also possible to launch a terminal inside the TeachingHub and you will find you files from your home directory there.
It might be that your course has a separate Notebook/Kernel for you, with installed packages and dependencies required for this specific course. However, these Notebooks/Kernels are only available during these lectures and might be removed in the future. So be aware of that. Note that you can easily change a kernel in your notebook. So no need to delete, copy or do anything. Your data/notebook information is safe.
-
+
diff --git a/build_docs.sh b/build_docs.sh
new file mode 100644
index 0000000000000000000000000000000000000000..963b0374f72c5da90f3104ab9d5294c8c0228501
--- /dev/null
+++ b/build_docs.sh
@@ -0,0 +1,20 @@
+#!/bin/bash
+# By MB
+# Build a mkdocs Documentation from existing Documentation
+# requires jupyter-nbconvert
+
+DOCS="./docs"
+
+# Go through all directories creating a similar structure in docs.
+
+# Add tags to Markdown files
+#---
+#tags:
+# - Python
+# - JET
+#---
+
+# Build pages
+
+mkdocs build
+# Copy to the correct location
\ No newline at end of file
diff --git a/deploy.sh b/deploy.sh
new file mode 100644
index 0000000000000000000000000000000000000000..57d5767bd922d604885c478042e58225bfde729c
--- /dev/null
+++ b/deploy.sh
@@ -0,0 +1,11 @@
+#!/bin/bash
+# By MB
+# Deploy documentation to web root
+
+if [ $EUID -ne 0 ]; then
+ echo "Run as root"
+ exit 1
+fi
+echo "Deploying to /var/www/html/docs"
+mkdir -p /var/www/html/docs
+mkdocs build -d /var/www/html/docs
diff --git a/mkdocs.yml b/mkdocs.yml
new file mode 100644
index 0000000000000000000000000000000000000000..b4a22c5ea2b040d4d4226b7233843f99914bf391
--- /dev/null
+++ b/mkdocs.yml
@@ -0,0 +1,28 @@
+site_name: Computer Resources @ IMGW
+# default is ./site/
+# could change this to /var/www/html/docs
+site_dir: "/tmp/cr-site/"
+docs_dir: '.'
+repo_url: https://gitlab.phaidra.org/imgw/computer-resources
+repo_name: IMGW/Computer-Resources
+theme:
+ name: material
+ palette:
+ scheme: uniwien
+plugins:
+ - tags
+ - same-dir
+ - search
+
+extra_css:
+ - mkdocs/stylesheets/extra.css
+markdown_extensions:
+ - pymdownx.highlight:
+ linenums: true
+ - pymdownx.inlinehilite
+ - pymdownx.snippets:
+ check_paths: true
+ - pymdownx.superfences
+ - pymdownx.tasklist
+ - pymdownx.details
+ - pymdownx.emoji
\ No newline at end of file
diff --git a/mkdocs/img/GPFS-jet.png b/mkdocs/img/GPFS-jet.png
new file mode 100644
index 0000000000000000000000000000000000000000..dd004e82a4eea3f18443ddbd5c5e5c0cc75e8868
Binary files /dev/null and b/mkdocs/img/GPFS-jet.png differ
diff --git a/mkdocs/img/envmodules.png b/mkdocs/img/envmodules.png
new file mode 100644
index 0000000000000000000000000000000000000000..f1a524a0d7b6461e26d8c92f7b6e15ed304afda7
Binary files /dev/null and b/mkdocs/img/envmodules.png differ
diff --git a/Documentation/jet-job1.png b/mkdocs/img/jet-job1.png
similarity index 100%
rename from Documentation/jet-job1.png
rename to mkdocs/img/jet-job1.png
diff --git a/Documentation/jet-job2.png b/mkdocs/img/jet-job2.png
similarity index 100%
rename from Documentation/jet-job2.png
rename to mkdocs/img/jet-job2.png
diff --git a/Documentation/jet-login.png b/mkdocs/img/jet-login.png
similarity index 100%
rename from Documentation/jet-login.png
rename to mkdocs/img/jet-login.png
diff --git a/Documentation/jupyterhub-logo.svg b/mkdocs/img/jupyterhub-logo.svg
similarity index 100%
rename from Documentation/jupyterhub-logo.svg
rename to mkdocs/img/jupyterhub-logo.svg
diff --git a/Documentation/logo_img2_color.png b/mkdocs/img/logo_img2_color.png
similarity index 100%
rename from Documentation/logo_img2_color.png
rename to mkdocs/img/logo_img2_color.png
diff --git a/Documentation/logo_uniwien.jpg b/mkdocs/img/logo_uniwien.jpg
similarity index 100%
rename from Documentation/logo_uniwien.jpg
rename to mkdocs/img/logo_uniwien.jpg
diff --git a/mkdocs/img/rocket.png b/mkdocs/img/rocket.png
new file mode 100644
index 0000000000000000000000000000000000000000..20a9ab83ee5bbebfae07a1b211bbe8f78bd7d686
Binary files /dev/null and b/mkdocs/img/rocket.png differ
diff --git a/Documentation/screen-cheatsheet.png b/mkdocs/img/screen-cheatsheet.png
similarity index 100%
rename from Documentation/screen-cheatsheet.png
rename to mkdocs/img/screen-cheatsheet.png
diff --git a/Documentation/teachinghub-authenticator-manual.png b/mkdocs/img/teachinghub-authenticator-manual.png
similarity index 100%
rename from Documentation/teachinghub-authenticator-manual.png
rename to mkdocs/img/teachinghub-authenticator-manual.png
diff --git a/Documentation/teachinghub-authenticator.png b/mkdocs/img/teachinghub-authenticator.png
similarity index 100%
rename from Documentation/teachinghub-authenticator.png
rename to mkdocs/img/teachinghub-authenticator.png
diff --git a/Documentation/teachinghub-qrcode.png b/mkdocs/img/teachinghub-qrcode.png
similarity index 100%
rename from Documentation/teachinghub-qrcode.png
rename to mkdocs/img/teachinghub-qrcode.png
diff --git a/Documentation/teachinghublogin.png b/mkdocs/img/teachinghublogin.png
similarity index 100%
rename from Documentation/teachinghublogin.png
rename to mkdocs/img/teachinghublogin.png
diff --git a/Documentation/teachinghubnotebooks.png b/mkdocs/img/teachinghubnotebooks.png
similarity index 100%
rename from Documentation/teachinghubnotebooks.png
rename to mkdocs/img/teachinghubnotebooks.png
diff --git a/Documentation/teachinghubnotebooks_kernels.png b/mkdocs/img/teachinghubnotebooks_kernels.png
similarity index 100%
rename from Documentation/teachinghubnotebooks_kernels.png
rename to mkdocs/img/teachinghubnotebooks_kernels.png
diff --git a/mkdocs/stylesheets/extra.css b/mkdocs/stylesheets/extra.css
new file mode 100644
index 0000000000000000000000000000000000000000..f4e7f9964223b327efa873322d1d2a3fb06dcb67
--- /dev/null
+++ b/mkdocs/stylesheets/extra.css
@@ -0,0 +1,5 @@
+[data-md-color-scheme="uniwien"] {
+ --md-primary-fg-color: #0063a6;
+ --md-primary-fg-color--light: #0574bf;
+ --md-primary-fg-color--dark: #004675;
+ }
\ No newline at end of file