From 6d9d62e35bf9638b68b5be33a5e03360d3348321 Mon Sep 17 00:00:00 2001
From: Michael Blaschek <michael.blaschek@univie.ac.at>
Date: Fri, 30 Apr 2021 15:54:39 +0200
Subject: [PATCH] Update README.md, Jet-Cluster.md, SSH-VPN-VNC/README.md,
Python/Your-First-Notebook-onJet_v2.ipynb, SRVX8.md, SRVX2.md, SRVX1.md files
---
Jet-Cluster.md | 109 ++--
Python/Your-First-Notebook-onJet_v2.ipynb | 715 ++++++++++++++++++++++
README.md | 148 -----
SRVX1.md | 76 +--
SRVX2.md | 23 +-
SRVX8.md | 12 +-
SSH-VPN-VNC/README.md | 24 +
7 files changed, 839 insertions(+), 268 deletions(-)
create mode 100644 Python/Your-First-Notebook-onJet_v2.ipynb
diff --git a/Jet-Cluster.md b/Jet-Cluster.md
index ee8cc28..94967bb 100644
--- a/Jet-Cluster.md
+++ b/Jet-Cluster.md
@@ -5,48 +5,14 @@
# Getting Started
-Welcome to the HPC @IMG @UNIVIE and please follow these steps to become a productive member of our department and make good use of the computer resources. Efficiency is keen.
+Welcome to the HPC @IMG @UNIVIE and please follow these steps to become a productive member of our department and make good use of the computer resources.
+**Efficiency is keen.**
1. Connect to Jet
2. Load environment (libraries, compilers, interpreter, tools)
3. Checkout Code, Program, Compile, Test
4. Submit to compute Nodes
-## SSH
-- [x] Terminal, Putty, ...
-- [x] Network access @UNIVIE
-
-Use a terminal or [Putty](https://www.putty.org/) to connect with SSH to the server
-```bash
-ssh [user]@jet01.img.univie.ac.at
-```
-Please replace `[user]` with the username given by the [sysadmin](mailto:michael.blaschek@univie.ac.at) and supply your password when asked for. This should give you access to the login node. If you are outside of the university network, then please goto [VPN](#VPN) below.
-
-## VPN
-- [x] `u:account`
-
-The jet cluster is only accessible from within the virtual network of the university of Vienna. Therefore access from outside has to be granted via the [VPN-Service](https://vpn.univie.ac.at). Go there and login with your `u:account` and download the *Big-IP Edge* client for you system.
-
-
-
-Links:
- * [ZID-VPN](https://vpn.univie.ac.at/f5-w-68747470733a2f2f7a69642e756e697669652e61632e6174$$/vpn/)
- * Linux (Ubuntu, Generic), Windows, Mac: [VPN user guides](https://vpn.univie.ac.at/f5-w-68747470733a2f2f7a69642e756e697669652e61632e6174$$/vpn/anleitungen/)
- * Arch based AUR package [AUR f5fpc](https://aur.archlinux.org/packages/f5fpc/)
-
-Follow the install instructions for Windows, Mac and Linux and make sure the software works.
-
-
-
-On Windows and MAc you get a nice gui that requires you to fill in the VPN server: `vpn.univie.ac.at` and username and password from the `u:account`. On Linux execute the following:
-```
-f5fpc -s -t vpn.univie.ac.at -u [user]
-```
-The status can be checked with `f5fpc --info`.
-
-## VNC
-The login nodes (`jet01` and `jet02`) allow to run a `VNC`-Server
-
-:construction:
+[How to SSH / VNC / VPN](SSH-VPN-VNC/README.md)
## Jupyterhub
<img src="https://jupyter.org/assets/hublogo.svg" width="300px">
@@ -62,8 +28,9 @@ Login with your jet-Credentials, choose a job and the jupyterlab will be launche
<img src="Documentation/jet-job2.png" width="500px">
-:construction:
-
+There are several kernels available and some help can be found:
+ - [Python/](Python/)
+ - [Tutorial on Jet](Python/Your-First-Notebook-onJet_v2.ipynb)
## User Quotas and Restrictions
@@ -93,11 +60,8 @@ Node Setup
| Name | Value |
| --- | --- |
| Product | ThinkSystem SR650 |
-| Distro | RedHatEnterprise 8.2 Ootpa |
-| Kernel | 4.18.0-147.el8.x86_64 GNU/Linux |
| Processor | Intel(R) Xeon(R) Gold 6148 CPU @ 2.40GHz |
| Cores | 2 CPU, 20 physical cores per CPU, total 80 logical CPU units |
-| BaseBoard | Lenovo -\[7X06CTO1WW\]- Intel Corporation C624 Series Chipset |
| CPU Time | 350 kh |
| Memory | 755 GB Total |
| Memory/Core | 18.9 GB |
@@ -109,8 +73,8 @@ Global file system (GPFS) is present on all nodes with about 1 PB (~1000 TB) of
The typcial installation of a intel-cluster has the INTEL Compiler suite (`intel-parallel-studio`) and the open source GNU Compilers installed. Based on these two different compilers (`intel`, `gnu`), there are usually two version of each scientific software.
Major Libraries:
- - OpenMPI
- - HDF5
+ - OpenMPI (3.1.6, 4.0.5)
+ - HDF5
- NetCDF (C, Fortran)
- ECCODES from [ECMWF](https://confluence.ecmwf.int/display/ECC)
- Math libraries e.g. intel-mkl, lapack,scalapack
@@ -122,36 +86,47 @@ These software libraries are usually handled by environment modules.
<img src="http://modules.sourceforge.net/modules_red.svg" width="300px">
## Currently installed modules
+
```bash
->>> module av
----------------------------------- /jetfs/spack/share/spack/modules/linux-rhel8-skylake_avx512 -----------------------------------
-anaconda2/2019.10-gcc-8.3.1-5pou6ji ncview/2.1.8-gcc-8.3.1-s2owtzw
-anaconda3/2019.10-gcc-8.3.1-tmy5mgp netcdf-c/4.7.4-gcc-8.3.1-fh4nn6k
-anaconda3/2020.07-gcc-8.3.1-weugqkf netcdf-c/4.7.4-intel-20.0.2-337uqtc
-cdo/1.9.8-gcc-8.3.1-ipgvzeh netcdf-fortran/4.5.3-gcc-8.3.1-kfd2vkj
-eccodes/2.18.0-gcc-8.3.1-s7clum3 netcdf-fortran/4.5.3-intel-20.0.2-irdm5gq
-eccodes/2.18.0-intel-20.0.2-6tadpgr netlib-lapack/3.8.0-gcc-8.3.1-ue37lic
-enstools/2020.11.dev-gcc-8.3.1-fes7kgo netlib-scalapack/2.1.0-gcc-8.3.1-pbkjymd
-gcc/8.3.1-gcc-8.3.1-pp3wjou openblas/0.3.10-gcc-8.3.1-ncess5c
-geos/3.8.1-gcc-8.3.1-o76leir openmpi/3.1.6-gcc-8.3.1-rk5av53
-hdf5/1.12.0-gcc-8.3.1-awl4atl openmpi/3.1.6-intel-20.0.2-ubasrpk
-hdf5/1.12.0-intel-20.0.2-ezeotzr openmpi/4.0.5-gcc-8.3.1-773ztsv
-intel-mkl/2020.3.279-gcc-8.3.1-5xeezjw openmpi/4.0.5-intel-20.0.2-4wfaaz4
-intel-mkl/2020.3.279-intel-20.0.2-m7bxged parallel-netcdf/1.12.1-gcc-8.3.1-gng2jcu
-intel-parallel-studio/composer.2020.2-intel-20.0.2-zuot22y parallel-netcdf/1.12.1-gcc-8.3.1-xxrhtxn
-julia/1.5.2-gcc-8.3.1-3iwgkf7 parallel-netcdf/1.12.1-intel-20.0.2-sgz3yqs
-libemos/4.5.9-gcc-8.3.1-h3lqu2n proj/7.1.0-gcc-8.3.1-xcjaco5
-miniconda2/4.7.12.1-gcc-8.3.1-zduqggv zlib/1.2.11-gcc-8.3.1-bbbpnzp
-miniconda3/4.8.2-gcc-8.3.1-3m7b6t2 zlib/1.2.11-intel-20.0.2-3h374ov
-nco/4.9.3-gcc-8.3.1-jtokrle
+$ module av
+------------------------------------ /jetfs/spack/share/spack/modules/linux-rhel8-haswell -------------------------------------
+intel-parallel-studio/composer.2017.7-intel-17.0.7-disfj2g
+
+--------------------------------- /jetfs/spack/share/spack/modules/linux-rhel8-skylake_avx512 ---------------------------------
+anaconda2/2019.10-gcc-8.3.1-5pou6ji nco/4.9.3-intel-20.0.2-dhlqiyo
+anaconda3/2019.10-gcc-8.3.1-tmy5mgp ncview/2.1.8-gcc-8.3.1-s2owtzw
+anaconda3/2020.07-gcc-8.3.1-weugqkf ncview/2.1.8-intel-20.0.2-3taqdda
+anaconda3/2020.11-gcc-8.3.1-gramgir netcdf-c/4.7.4-gcc-8.3.1-fh4nn6k
+cdo/1.9.8-gcc-8.3.1-ipgvzeh netcdf-c/4.7.4-gcc-8.3.1-MPI3.1.6-y5gknpt
+eccodes/2.18.0-gcc-8.3.1-s7clum3 netcdf-c/4.7.4-intel-20.0.2-337uqtc
+eccodes/2.18.0-intel-20.0.2-6tadpgr netcdf-fortran/4.5.3-gcc-8.3.1-kfd2vkj
+enstools/2020.11.dev-gcc-8.3.1-fes7kgo netcdf-fortran/4.5.3-gcc-8.3.1-MPI3.1.6-rjcxqb6
+esmf/7.1.0r-gcc-8.3.1-4fijz4q netcdf-fortran/4.5.3-intel-20.0.2-irdm5gq
+gcc/8.3.1-gcc-8.3.1-pp3wjou netlib-lapack/3.8.0-gcc-8.3.1-ue37lic
+geos/3.8.1-gcc-8.3.1-o76leir netlib-scalapack/2.1.0-gcc-8.3.1-pbkjymd
+hdf5/1.10.7-gcc-8.3.1-MPI3.1.6-vm3avor openblas/0.3.10-gcc-8.3.1-ncess5c
+hdf5/1.12.0-gcc-8.3.1-awl4atl openmpi/3.1.6-gcc-8.3.1-rk5av53
+hdf5/1.12.0-intel-20.0.2-ezeotzr openmpi/3.1.6-intel-20.0.2-ubasrpk
+intel-mkl/2020.3.279-gcc-8.3.1-5xeezjw openmpi/4.0.5-gcc-8.3.1-773ztsv
+intel-mkl/2020.3.279-intel-20.0.2-m7bxged openmpi/4.0.5-intel-20.0.2-4wfaaz4
+intel-oneapi-compilers/2021.2.0-oneapi-2021.2.0-6kdzddx openmpi/4.0.5-oneapi-2021.2.0-hrfsxrd
+intel-oneapi-mpi/2021.2.0-oneapi-2021.2.0-haqpxfl parallel-netcdf/1.12.1-gcc-8.3.1-MPI3.1.6-gng2jcu
+intel-parallel-studio/composer.2020.2-intel-20.0.2-zuot22y parallel-netcdf/1.12.1-gcc-8.3.1-xxrhtxn
+julia/1.5.2-gcc-8.3.1-3iwgkf7 parallel-netcdf/1.12.1-intel-20.0.2-sgz3yqs
+libemos/4.5.9-gcc-8.3.1-h3lqu2n perl/5.32.0-intel-20.0.2-2d23x7l
+miniconda2/4.7.12.1-gcc-8.3.1-zduqggv proj/7.1.0-gcc-8.3.1-xcjaco5
+miniconda3/4.8.2-gcc-8.3.1-3m7b6t2 zlib/1.2.11-gcc-8.3.1-bbbpnzp
+ncl/6.6.2-gcc-8.3.1-MPI3.1.6-3dxuv5f zlib/1.2.11-intel-20.0.2-3h374ov
+nco/4.9.3-gcc-8.3.1-g7o6lao
```
+
Using [environment modules](https://modules.readthedocs.io/en/latest/) it is possible to have different software libraries (versions, compilers) side-by-side and ready to be loaded. Be aware that some libraries are dependent on others. It is recommended to load the highest rank library first to check what dependencies are loaded as well. e.g.:
```
->>> module load eccodes/2.18.0-intel-20.0.2-6tadpgr
+$ module load eccodes/2.18.0-intel-20.0.2-6tadpgr
```
loads the `ECCODES` library and all dependencies. e.g. intel compilers, as indicated by the naming.
```
->>> module list
+$ module list
Currently Loaded Modulefiles:
1) zlib/1.2.11-intel-20.0.2-3h374ov 3) hdf5/1.12.0-intel-20.0.2-ezeotzr 5) netcdf-c/4.7.4-intel-20.0.2-337uqtc
2) openmpi/4.0.5-intel-20.0.2-4wfaaz4 4) parallel-netcdf/1.12.1-intel-20.0.2-sgz3yqs 6) eccodes/2.18.0-intel-20.0.2-6tadpgr
diff --git a/Python/Your-First-Notebook-onJet_v2.ipynb b/Python/Your-First-Notebook-onJet_v2.ipynb
new file mode 100644
index 0000000..582ffc4
--- /dev/null
+++ b/Python/Your-First-Notebook-onJet_v2.ipynb
@@ -0,0 +1,715 @@
+{
+ "cells": [
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "# Welcome to Jet\n",
+ "\n",
+ "<img src='https://homepage.univie.ac.at/michael.blaschek/openfiles/rocket.png' width='100px'><img src='https://homepage.univie.ac.at/michael.blaschek/openfiles/uni.jpg' width='400px'>\n",
+ "\n",
+ "You have done it! Your Notebook Server is running on the Jet Cluster.\n",
+ "\n",
+ "Welcome, please skip this, if your know where you are, as this is meant as an introduction. However, not complete as it is. \n",
+ "If you need more infos about Notebooks have a look at [my Homepage](https://homepage.univie.ac.at/michael.blaschek/#python) and there are numerous Tutorials on the Internet for any advice, but feel free to ask me and give me feedback.\n",
+ "\n",
+ "Help can be found here:\n",
+ "* Goto Help -> JupyterLab Reference\n",
+ "* Goto Help -> Jupyter Reference\n",
+ "* [Tour](https://jupyterlab.readthedocs.io/en/stable/user/interface.html)\n",
+ "\n",
+ "\n",
+ "In the following I will show you some useful things.\n",
+ "\n",
+ "First: This Notebook is in your Home directory, but also on `/jetfs/scratch` for any reason."
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "## Check your current Host"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 1,
+ "metadata": {},
+ "outputs": [
+ {
+ "name": "stdout",
+ "output_type": "stream",
+ "text": [
+ " Static hostname: localhost.localdomain\n",
+ "Transient hostname: jet03.jet.local\n",
+ " Icon name: computer-server\n",
+ " Chassis: server\n",
+ " Operating System: \u001b]8;;https://www.centos.org/\u0007CentOS Linux 8 (Core)\u001b]8;;\u0007\n",
+ " CPE OS Name: cpe:/o:centos:centos:8\n",
+ " Kernel: Linux 4.18.0-147.el8.x86_64\n",
+ " Architecture: x86-64\n"
+ ]
+ }
+ ],
+ "source": [
+ "!hostnamectl"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "## Who is running this process"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 2,
+ "metadata": {},
+ "outputs": [
+ {
+ "name": "stdout",
+ "output_type": "stream",
+ "text": [
+ "mblaschek\n"
+ ]
+ }
+ ],
+ "source": [
+ "!whoami"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 3,
+ "metadata": {},
+ "outputs": [
+ {
+ "name": "stdout",
+ "output_type": "stream",
+ "text": [
+ "/jetfs/home/mblaschek/Documents\n"
+ ]
+ }
+ ],
+ "source": [
+ "# Where are you\n",
+ "!pwd"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 4,
+ "metadata": {},
+ "outputs": [
+ {
+ "name": "stdout",
+ "output_type": "stream",
+ "text": [
+ "Fri Oct 16 23:10:58 2020\n",
+ " JOBID PARTITION NAME USER STATE TIME TIME_LIMI NODES NODELIST(REASON) \n",
+ " 235 compute jupyters mblasche RUNNING 6:29:07 8:00:00 1 jet03 \n"
+ ]
+ }
+ ],
+ "source": [
+ "# Show my Current Jobs in the Queue\n",
+ "!squeue -u $(id -u) -l"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "There you can see your `TIME_LIMIT` of 8 hours again and how much time has passed on your current jobs"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 5,
+ "metadata": {},
+ "outputs": [
+ {
+ "name": "stdout",
+ "output_type": "stream",
+ "text": [
+ "/jetfs/spack/opt/spack/linux-rhel8-skylake_avx512/gcc-8.3.1/miniconda3-4.8.2-3m7b6t2kgedyr3jnd2nasmgiq7wm27iv/bin/python\n"
+ ]
+ }
+ ],
+ "source": [
+ "# Import sys and check the executable (so what python you are using)\n",
+ "import sys\n",
+ "print(sys.executable)"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "**Notice the funny path?**\n",
+ "\n",
+ " Yes this is SPACK your nice scientific package manager. \n",
+ " Allows to build custom libraries and applications tailored to our needs!\n",
+ " Please not that software currently installed in /opt on JET01 is not available here.\n",
+ " Software under /opt is deprecated and will be removed after a full transition \n",
+ " to the current libraries\n",
+ "\n",
+ "More information on [SPACK](https://spack.io/) \n",
+ "\n",
+ "## Check your environmental PATH Variable, could be useful to know"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 6,
+ "metadata": {},
+ "outputs": [
+ {
+ "data": {
+ "text/plain": [
+ "'/jetfs/spack/opt/spack/linux-rhel8-skylake_avx512/gcc-8.3.1/miniconda3-4.8.2-3m7b6t2kgedyr3jnd2nasmgiq7wm27iv/bin:/bin:/jetfs/spack/bin:/jetfs/userservices:/opt/jupyterhub/bin:/opt/slurm/bin:/sbin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/share/Modules/bin::/jetfs/home/mblaschek/bin:/jetfs/home/mblaschek/.local/bin'"
+ ]
+ },
+ "execution_count": 6,
+ "metadata": {},
+ "output_type": "execute_result"
+ }
+ ],
+ "source": [
+ "%env PATH"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "## Check what modules are currently loaded in this Notebook session"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 7,
+ "metadata": {},
+ "outputs": [
+ {
+ "name": "stdout",
+ "output_type": "stream",
+ "text": [
+ "Currently Loaded Modulefiles:\n",
+ " 1) miniconda3/4.8.2-gcc-8.3.1-3m7b6t2 \n"
+ ]
+ }
+ ],
+ "source": [
+ "!module list --no-pager"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "Put this file `load_modules_into_jupyter.conf` into your Home directory and your Notebook environment will have these modules loaded.\n",
+ "It is not possible to load these modules via magic (python magic ;).\n",
+ "**It needs the empty line at the end...**"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 8,
+ "metadata": {},
+ "outputs": [
+ {
+ "name": "stdout",
+ "output_type": "stream",
+ "text": [
+ "Writing /jetfs/home/mblaschek/load_modules_into_jupyter.conf\n"
+ ]
+ }
+ ],
+ "source": [
+ "%%writefile ~/load_modules_into_jupyter.conf\n",
+ "eccodes/2.18.0-gcc-8.3.1-s7clum3\n"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 9,
+ "metadata": {},
+ "outputs": [
+ {
+ "name": "stdout",
+ "output_type": "stream",
+ "text": [
+ "eccodes/2.18.0-gcc-8.3.1-s7clum3\n"
+ ]
+ }
+ ],
+ "source": [
+ "!cat ~/load_modules_into_jupyter.conf"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "**Restarting the kernel will show now loaded modules**"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 1,
+ "metadata": {},
+ "outputs": [
+ {
+ "name": "stdout",
+ "output_type": "stream",
+ "text": [
+ "Currently Loaded Modulefiles:\n",
+ " 1) miniconda3/4.8.2-gcc-8.3.1-3m7b6t2 \n",
+ " 2) openmpi/4.0.5-gcc-8.3.1-773ztsv \n",
+ " 3) hdf5/1.12.0-gcc-8.3.1-awl4atl \n",
+ " 4) parallel-netcdf/1.12.1-gcc-8.3.1-xxrhtxn \n",
+ " 5) netcdf-c/4.7.4-gcc-8.3.1-fh4nn6k \n",
+ " 6) eccodes/2.18.0-gcc-8.3.1-s7clum3 \n"
+ ]
+ }
+ ],
+ "source": [
+ "# after restarting\n",
+ "!module list --no-pager"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "## Currently available Modules"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 11,
+ "metadata": {},
+ "outputs": [
+ {
+ "name": "stdout",
+ "output_type": "stream",
+ "text": [
+ "--------- /jetfs/spack/share/spack/modules/linux-rhel8-skylake_avx512 ----------\n",
+ "anaconda2/2019.10-gcc-8.3.1-5pou6ji \n",
+ "anaconda3/2019.10-gcc-8.3.1-tmy5mgp \n",
+ "anaconda3/2020.07-gcc-8.3.1-weugqkf \n",
+ "cdo/1.9.8-gcc-8.3.1-ipgvzeh \n",
+ "eccodes/2.18.0-gcc-8.3.1-s7clum3 \n",
+ "hdf5/1.12.0-gcc-8.3.1-awl4atl \n",
+ "intel-mkl/2020.3.279-gcc-8.3.1-5xeezjw \n",
+ "miniconda2/4.7.12.1-gcc-8.3.1-zduqggv \n",
+ "miniconda3/4.8.2-gcc-8.3.1-3m7b6t2 \n",
+ "netcdf-c/4.7.4-gcc-8.3.1-fh4nn6k \n",
+ "netcdf-fortran/4.5.3-gcc-8.3.1-kfd2vkj \n",
+ "openmpi/3.1.6-gcc-8.3.1-rk5av53 \n",
+ "openmpi/4.0.5-gcc-8.3.1-773ztsv \n",
+ "parallel-netcdf/1.12.1-gcc-8.3.1-gng2jcu \n",
+ "parallel-netcdf/1.12.1-gcc-8.3.1-xxrhtxn \n",
+ "\n",
+ "--------- /jetfs/spack/share/spack/modules/linux-rhel8-skylake_avx512 ----------\n",
+ "anaconda2/2019.10-gcc-8.3.1-5pou6ji \n",
+ "anaconda3/2019.10-gcc-8.3.1-tmy5mgp \n",
+ "anaconda3/2020.07-gcc-8.3.1-weugqkf \n",
+ "cdo/1.9.8-gcc-8.3.1-ipgvzeh \n",
+ "eccodes/2.18.0-gcc-8.3.1-s7clum3 \n",
+ "hdf5/1.12.0-gcc-8.3.1-awl4atl \n",
+ "intel-mkl/2020.3.279-gcc-8.3.1-5xeezjw \n",
+ "miniconda2/4.7.12.1-gcc-8.3.1-zduqggv \n",
+ "miniconda3/4.8.2-gcc-8.3.1-3m7b6t2 \n",
+ "netcdf-c/4.7.4-gcc-8.3.1-fh4nn6k \n",
+ "netcdf-fortran/4.5.3-gcc-8.3.1-kfd2vkj \n",
+ "openmpi/3.1.6-gcc-8.3.1-rk5av53 \n",
+ "openmpi/4.0.5-gcc-8.3.1-773ztsv \n",
+ "parallel-netcdf/1.12.1-gcc-8.3.1-gng2jcu \n",
+ "parallel-netcdf/1.12.1-gcc-8.3.1-xxrhtxn \n"
+ ]
+ }
+ ],
+ "source": [
+ "# --no-pager is needed only inside notebook\n",
+ "!module avail --no-pager"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "# Software on JET\n",
+ "Currently:\n",
+ " - [x] Anaconda (2,3)\n",
+ " - [x] Miniconda (2,3)\n",
+ " - [x] OPENMPI (4.0.5, 3.6.1) optimized for JET\n",
+ " - [x] HDF5 (1.12) optimized with OPENMPI\n",
+ " - [x] NetCDF (Parallel (NC3), NC4->HDF5)\n",
+ " - [x] Eccodes (2.18)\n",
+ " - [x] CDO\n",
+ " - [ ] Intel compiled Libraries (License) \n",
+ " - [ ] Emoslib \n",
+ " - [ ] Ecaccess\n",
+ " - [ ] RTTOV"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "## Currently there are a few choices:\n",
+ "\n",
+ "1. Python (2, 3)\n",
+ " * Anaconda\n",
+ " * Miniconda\n",
+ "2. Julia (not yet)\n",
+ "3. R (not yet)"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "## Install Python packages"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 12,
+ "metadata": {},
+ "outputs": [
+ {
+ "name": "stdout",
+ "output_type": "stream",
+ "text": [
+ "Requirement already satisfied: numpy in /jetfs/home/mblaschek/.local/lib/python3.8/site-packages (1.19.2)\n",
+ "Requirement already satisfied: scipy in /jetfs/home/mblaschek/.local/lib/python3.8/site-packages (1.5.2)\n",
+ "Note: you may need to restart the kernel to use updated packages.\n"
+ ]
+ }
+ ],
+ "source": [
+ "# Please use the --user flag to install the packages into .local/lib/python...\n",
+ "%pip install --user numpy scipy"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 13,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "import os\n",
+ "import sys\n",
+ "# Need to update search Path / only necessary here for the first time\n",
+ "sys.path.append(os.path.expandvars(\"$HOME/.local/lib/python3.8/site-packages\"))"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 14,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "import numpy as np\n",
+ "import scipy as sp\n",
+ "\n",
+ "# WORKS! ;)"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "## Using conda"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 15,
+ "metadata": {},
+ "outputs": [
+ {
+ "name": "stdout",
+ "output_type": "stream",
+ "text": [
+ "\n",
+ " active environment : None\n",
+ " user config file : /jetfs/home/mblaschek/.condarc\n",
+ " populated config files : /jetfs/home/mblaschek/.condarc\n",
+ " conda version : 4.8.5\n",
+ " conda-build version : not installed\n",
+ " python version : 3.8.1.final.0\n",
+ " virtual packages : __glibc=2.28\n",
+ " base environment : /jetfs/spack/opt/spack/linux-rhel8-skylake_avx512/gcc-8.3.1/miniconda3-4.8.2-3m7b6t2kgedyr3jnd2nasmgiq7wm27iv (read only)\n",
+ " channel URLs : https://repo.anaconda.com/pkgs/main/linux-64\n",
+ " https://repo.anaconda.com/pkgs/main/noarch\n",
+ " https://repo.anaconda.com/pkgs/r/linux-64\n",
+ " https://repo.anaconda.com/pkgs/r/noarch\n",
+ " package cache : /jetfs/spack/opt/spack/linux-rhel8-skylake_avx512/gcc-8.3.1/miniconda3-4.8.2-3m7b6t2kgedyr3jnd2nasmgiq7wm27iv/pkgs\n",
+ " /jetfs/home/mblaschek/.conda/pkgs\n",
+ " envs directories : /jetfs/home/mblaschek/.conda/envs\n",
+ " /jetfs/spack/opt/spack/linux-rhel8-skylake_avx512/gcc-8.3.1/miniconda3-4.8.2-3m7b6t2kgedyr3jnd2nasmgiq7wm27iv/envs\n",
+ " platform : linux-64\n",
+ " user-agent : conda/4.8.5 requests/2.22.0 CPython/3.8.1 Linux/4.18.0-147.el8.x86_64 centos/8.2.2004 glibc/2.28\n",
+ " UID:GID : 54212:100\n",
+ " netrc file : None\n",
+ " offline mode : False\n",
+ "\n"
+ ]
+ }
+ ],
+ "source": [
+ "!conda info"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 16,
+ "metadata": {},
+ "outputs": [
+ {
+ "name": "stdout",
+ "output_type": "stream",
+ "text": [
+ "Collecting package metadata (current_repodata.json): done\n",
+ "Solving environment: failed with repodata from current_repodata.json, will retry with next repodata source.\n",
+ "Collecting package metadata (repodata.json): done\n",
+ "Solving environment: done\n",
+ "\n",
+ "## Package Plan ##\n",
+ "\n",
+ " environment location: /jetfs/home/mblaschek/.conda/envs/myenv\n",
+ "\n",
+ " added / updated specs:\n",
+ " - ipykernel\n",
+ " - numpy\n",
+ " - python=3.7\n",
+ "\n",
+ "\n",
+ "The following NEW packages will be INSTALLED:\n",
+ "\n",
+ " _libgcc_mutex pkgs/main/linux-64::_libgcc_mutex-0.1-main\n",
+ " backcall pkgs/main/noarch::backcall-0.2.0-py_0\n",
+ " blas pkgs/main/linux-64::blas-1.0-mkl\n",
+ " ca-certificates pkgs/main/linux-64::ca-certificates-2020.10.14-0\n",
+ " certifi pkgs/main/linux-64::certifi-2020.6.20-py37_0\n",
+ " decorator pkgs/main/noarch::decorator-4.4.2-py_0\n",
+ " intel-openmp pkgs/main/linux-64::intel-openmp-2020.2-254\n",
+ " ipykernel pkgs/main/linux-64::ipykernel-5.3.4-py37h5ca1d4c_0\n",
+ " ipython pkgs/main/linux-64::ipython-7.18.1-py37h5ca1d4c_0\n",
+ " ipython_genutils pkgs/main/linux-64::ipython_genutils-0.2.0-py37_0\n",
+ " jedi pkgs/main/linux-64::jedi-0.17.2-py37_0\n",
+ " jupyter_client pkgs/main/noarch::jupyter_client-6.1.7-py_0\n",
+ " jupyter_core pkgs/main/linux-64::jupyter_core-4.6.3-py37_0\n",
+ " ld_impl_linux-64 pkgs/main/linux-64::ld_impl_linux-64-2.33.1-h53a641e_7\n",
+ " libedit pkgs/main/linux-64::libedit-3.1.20191231-h14c3975_1\n",
+ " libffi pkgs/main/linux-64::libffi-3.3-he6710b0_2\n",
+ " libgcc-ng pkgs/main/linux-64::libgcc-ng-9.1.0-hdf63c60_0\n",
+ " libsodium pkgs/main/linux-64::libsodium-1.0.18-h7b6447c_0\n",
+ " libstdcxx-ng pkgs/main/linux-64::libstdcxx-ng-9.1.0-hdf63c60_0\n",
+ " mkl pkgs/main/linux-64::mkl-2020.2-256\n",
+ " mkl-service pkgs/main/linux-64::mkl-service-2.3.0-py37he904b0f_0\n",
+ " mkl_fft pkgs/main/linux-64::mkl_fft-1.2.0-py37h23d657b_0\n",
+ " mkl_random pkgs/main/linux-64::mkl_random-1.1.1-py37h0573a6f_0\n",
+ " ncurses pkgs/main/linux-64::ncurses-6.2-he6710b0_1\n",
+ " numpy pkgs/main/linux-64::numpy-1.19.1-py37hbc911f0_0\n",
+ " numpy-base pkgs/main/linux-64::numpy-base-1.19.1-py37hfa32c7d_0\n",
+ " openssl pkgs/main/linux-64::openssl-1.1.1h-h7b6447c_0\n",
+ " parso pkgs/main/noarch::parso-0.7.0-py_0\n",
+ " pexpect pkgs/main/linux-64::pexpect-4.8.0-py37_1\n",
+ " pickleshare pkgs/main/linux-64::pickleshare-0.7.5-py37_1001\n",
+ " pip pkgs/main/linux-64::pip-20.2.3-py37_0\n",
+ " prompt-toolkit pkgs/main/noarch::prompt-toolkit-3.0.8-py_0\n",
+ " ptyprocess pkgs/main/linux-64::ptyprocess-0.6.0-py37_0\n",
+ " pygments pkgs/main/noarch::pygments-2.7.1-py_0\n",
+ " python pkgs/main/linux-64::python-3.7.9-h7579374_0\n",
+ " python-dateutil pkgs/main/noarch::python-dateutil-2.8.1-py_0\n",
+ " pyzmq pkgs/main/linux-64::pyzmq-19.0.2-py37he6710b0_1\n",
+ " readline pkgs/main/linux-64::readline-8.0-h7b6447c_0\n",
+ " setuptools pkgs/main/linux-64::setuptools-50.3.0-py37hb0f4dca_1\n",
+ " six pkgs/main/noarch::six-1.15.0-py_0\n",
+ " sqlite pkgs/main/linux-64::sqlite-3.33.0-h62c20be_0\n",
+ " tk pkgs/main/linux-64::tk-8.6.10-hbc83047_0\n",
+ " tornado pkgs/main/linux-64::tornado-6.0.4-py37h7b6447c_1\n",
+ " traitlets pkgs/main/noarch::traitlets-5.0.5-py_0\n",
+ " wcwidth pkgs/main/noarch::wcwidth-0.2.5-py_0\n",
+ " wheel pkgs/main/noarch::wheel-0.35.1-py_0\n",
+ " xz pkgs/main/linux-64::xz-5.2.5-h7b6447c_0\n",
+ " zeromq pkgs/main/linux-64::zeromq-4.3.3-he6710b0_3\n",
+ " zlib pkgs/main/linux-64::zlib-1.2.11-h7b6447c_3\n",
+ "\n",
+ "\n",
+ "Preparing transaction: done\n",
+ "Verifying transaction: done\n",
+ "Executing transaction: done\n",
+ "#\n",
+ "# To activate this environment, use\n",
+ "#\n",
+ "# $ conda activate myenv\n",
+ "#\n",
+ "# To deactivate an active environment, use\n",
+ "#\n",
+ "# $ conda deactivate\n",
+ "\n"
+ ]
+ }
+ ],
+ "source": [
+ "# Create a environment with conda\n",
+ "!conda create -y -n myenv python=3.7 numpy ipykernel"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 17,
+ "metadata": {},
+ "outputs": [
+ {
+ "name": "stdout",
+ "output_type": "stream",
+ "text": [
+ "Installed kernelspec MYENV in /jetfs/home/mblaschek/.local/share/jupyter/kernels/myenv\n"
+ ]
+ }
+ ],
+ "source": [
+ "!~/.conda/envs/myenv/bin/python -m ipykernel install --user --name MYENV --display-name \"MYCONDA\""
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 18,
+ "metadata": {},
+ "outputs": [
+ {
+ "name": "stdout",
+ "output_type": "stream",
+ "text": [
+ "Available kernels:\n",
+ " myenv /jetfs/home/mblaschek/.local/share/jupyter/kernels/myenv\n",
+ " python3 /jetfs/spack/opt/spack/linux-rhel8-skylake_avx512/gcc-8.3.1/miniconda3-4.8.2-3m7b6t2kgedyr3jnd2nasmgiq7wm27iv/share/jupyter/kernels/python3\n"
+ ]
+ }
+ ],
+ "source": [
+ "!jupyter kernelspec list"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "## Custom Environment Python Environment"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 19,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "# Setup a virtual Environment for your Python in the directoy: myfancyenv\n",
+ "!python -m venv myfancyenv "
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 20,
+ "metadata": {},
+ "outputs": [
+ {
+ "name": "stdout",
+ "output_type": "stream",
+ "text": [
+ "\u001b[33mWARNING: You are using pip version 19.2.3, however version 20.2.3 is available.\n",
+ "You should consider upgrading via the 'pip install --upgrade pip' command.\u001b[0m\n"
+ ]
+ }
+ ],
+ "source": [
+ "# Install some package into that environment\n",
+ "# important is ipykernel to use it within jupyterlab/notebook\n",
+ "!./myfancyenv/bin/pip -q install numpy scipy pandas ipykernel"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 21,
+ "metadata": {},
+ "outputs": [
+ {
+ "name": "stdout",
+ "output_type": "stream",
+ "text": [
+ "Installed kernelspec fancy in /jetfs/home/mblaschek/.local/share/jupyter/kernels/fancy\n"
+ ]
+ }
+ ],
+ "source": [
+ "# Allow this environment to be available on the LAUNCHERS (+) sign on the left\n",
+ "!./myfancyenv/bin/python -m ipykernel install --user --name fancy --display-name \"My Python\""
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "Wait a bit and the new python environment should be available on your launchers."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 22,
+ "metadata": {},
+ "outputs": [
+ {
+ "name": "stdout",
+ "output_type": "stream",
+ "text": [
+ "Available kernels:\n",
+ " fancy /jetfs/home/mblaschek/.local/share/jupyter/kernels/fancy\n",
+ " myenv /jetfs/home/mblaschek/.local/share/jupyter/kernels/myenv\n",
+ " python3 /jetfs/spack/opt/spack/linux-rhel8-skylake_avx512/gcc-8.3.1/miniconda3-4.8.2-3m7b6t2kgedyr3jnd2nasmgiq7wm27iv/share/jupyter/kernels/python3\n"
+ ]
+ }
+ ],
+ "source": [
+ "!jupyter kernelspec list"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "There you go. Your personal Environment.\n",
+ "\n",
+ "# If you need help. Feel free to ask me. Enjoy!"
+ ]
+ }
+ ],
+ "metadata": {
+ "kernelspec": {
+ "display_name": "Miniconda3 (4.8.2)",
+ "language": "python",
+ "name": "python3"
+ },
+ "language_info": {
+ "codemirror_mode": {
+ "name": "ipython",
+ "version": 3
+ },
+ "file_extension": ".py",
+ "mimetype": "text/x-python",
+ "name": "python",
+ "nbconvert_exporter": "python",
+ "pygments_lexer": "ipython3",
+ "version": "3.8.1"
+ }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 4
+}
diff --git a/README.md b/README.md
index 8445bcf..5d73678 100644
--- a/README.md
+++ b/README.md
@@ -25,50 +25,6 @@ Missing solutions are very welcome.
[Go to Description](Jet-Cluster.md)
-### Architecture
-
-| Name | Value |
-| --- | --- |
-| Product | ThinkSystem SR650 |
-| Distro | RedHatEnterprise 8.2 Ootpa |
-| Kernel | 4.18.0-147.el8.x86_64 GNU/Linux |
-| Processor | Intel(R) Xeon(R) Gold 6148 CPU @ 2.40GHz |
-| Cores | 2 CPU, 20 physical cores per CPU, total 80 logical CPU units |
-| BaseBoard | Lenovo -\[7X06CTO1WW\]- Intel Corporation C624 Series Chipset |
-| CPU Time | 350 kh |
-| Memory | 755 GB Total |
-| Memory/Core | 18.9 GB |
-| Network | 40 Gbit/s (Infiniband) |
-
-
-### Scientific Software
- - GNU compilers, Intel compilers, OpenMPI, HDF5, NetCDF, Eccodes, ...
- - Anaconda Python, Julia
-
-```bash
->>> module av
----------------------------------- /jetfs/spack/share/spack/modules/linux-rhel8-skylake_avx512 -----------------------------------
-anaconda2/2019.10-gcc-8.3.1-5pou6ji ncview/2.1.8-gcc-8.3.1-s2owtzw
-anaconda3/2019.10-gcc-8.3.1-tmy5mgp netcdf-c/4.7.4-gcc-8.3.1-fh4nn6k
-anaconda3/2020.07-gcc-8.3.1-weugqkf netcdf-c/4.7.4-intel-20.0.2-337uqtc
-cdo/1.9.8-gcc-8.3.1-ipgvzeh netcdf-fortran/4.5.3-gcc-8.3.1-kfd2vkj
-eccodes/2.18.0-gcc-8.3.1-s7clum3 netcdf-fortran/4.5.3-intel-20.0.2-irdm5gq
-eccodes/2.18.0-intel-20.0.2-6tadpgr netlib-lapack/3.8.0-gcc-8.3.1-ue37lic
-enstools/2020.11.dev-gcc-8.3.1-fes7kgo netlib-scalapack/2.1.0-gcc-8.3.1-pbkjymd
-gcc/8.3.1-gcc-8.3.1-pp3wjou openblas/0.3.10-gcc-8.3.1-ncess5c
-geos/3.8.1-gcc-8.3.1-o76leir openmpi/3.1.6-gcc-8.3.1-rk5av53
-hdf5/1.12.0-gcc-8.3.1-awl4atl openmpi/3.1.6-intel-20.0.2-ubasrpk
-hdf5/1.12.0-intel-20.0.2-ezeotzr openmpi/4.0.5-gcc-8.3.1-773ztsv
-intel-mkl/2020.3.279-gcc-8.3.1-5xeezjw openmpi/4.0.5-intel-20.0.2-4wfaaz4
-intel-mkl/2020.3.279-intel-20.0.2-m7bxged parallel-netcdf/1.12.1-gcc-8.3.1-gng2jcu
-intel-parallel-studio/composer.2020.2-intel-20.0.2-zuot22y parallel-netcdf/1.12.1-gcc-8.3.1-xxrhtxn
-julia/1.5.2-gcc-8.3.1-3iwgkf7 parallel-netcdf/1.12.1-intel-20.0.2-sgz3yqs
-libemos/4.5.9-gcc-8.3.1-h3lqu2n proj/7.1.0-gcc-8.3.1-xcjaco5
-miniconda2/4.7.12.1-gcc-8.3.1-zduqggv zlib/1.2.11-gcc-8.3.1-bbbpnzp
-miniconda3/4.8.2-gcc-8.3.1-3m7b6t2 zlib/1.2.11-intel-20.0.2-3h374ov
-nco/4.9.3-gcc-8.3.1-jtokrle
-```
-
### JupyterHub
[JET01-Jupyterhub](https://jet01.img.univie.ac.at)
@@ -77,33 +33,6 @@ Teaching and webaccess server
[Go to Description](SRVX1.md)
-### Architecture
-| Name | Value |
-| --- | --- |
-| Product | PowerEdge R720xd |
-| Distro | CentOS 6.10 Final |
-| Kernel | 2.6.32-754.29.2.el6.centos.plus.x86_64 GNU/Linux |
-| Processor | Intel(R) Xeon(R) CPU E5-2690 0 @ 2.90GHz |
-| Cores | 2 CPU, 8 physical cores per CPU, total 32 logical CPU units |
-| BaseBoard | Dell Inc. 0C4Y3R Intel Corporation C600/X79 series chipset |
-| CPU time | 140 kh |
-| Memory | 190 GB Total |
-| Memory/Core | 11.9 GB |
-| Network | 10 Gbit/s |
-
-### Scientific Software
- - GNU compilers, OpenMPI, HDF5, NetCDF, Eccodes, ...
- - Anaconda Python
-
-```bash
->>> module av
-------------- /home/opt/spack/share/spack/modules/linux-centos6-sandybridge -------------
-anaconda3/2020.07-gcc-5.3.0 hdf5/1.10.7-gcc-5.3.0 openmpi/3.1.6-gcc-5.3.0
-eccodes/2.18.0-gcc-5.3.0 miniconda3/4.8.2-gcc-5.3.0 proj/7.1.0-gcc-5.3.0
-gcc/5.3.0-gcc-5.3.0 netcdf-c/4.7.4-gcc-5.3.0 zlib/1.2.11-gcc-5.3.0
-git/2.29.0-gcc-5.3.0 netcdf-fortran/4.5.3-gcc-5.3.0
-```
-
### JupyterHub
Direct Access: [Goto SRVX1-Jupyterhub](https://srvx1.img.univie.ac.at)
@@ -114,64 +43,11 @@ Computing Node @UZA2
[Go to Description](SRVX2.md)
-### Architecture
-| Name | Value |
-| --- | --- |
-| Product | PowerEdge R940 |
-| Distro | CentOS 8.2.2004 Core |
-| Kernel | 4.18.0-147.el8.x86_64 GNU/Linux |
-| Processor | Intel(R) Xeon(R) Gold 6148 CPU @ 2.40GHz |
-| Cores | 4 CPU, 20 physical cores per CPU, total 160 logical CPU units |
-| BaseBoard | Dell Inc. 0D41HC Intel Corporation C621 Series Chipset |
-| CPU time | 700 kh |
-| Memory | 376 GB Total |
-| Memory/Core | 9.4 GB |
-### Software
- - GNU compilers, Intel compilers, OpenMPI, HDF5, NetCDF, Eccodes, ...
- - Anaconda Python
-
-```bash
->>> module av
------------ /home/spack-root/share/spack/modules/linux-centos8-skylake_avx512 -----------
-anaconda3/2020.07-gcc-8.3.1 miniconda3/4.8.2-gcc-8.3.1
-eccodes/2.18.0-intel-20.0.2 netcdf-c/4.7.4-gcc-8.3.1
-eccodes/2.19.1-gcc-8.3.1 netcdf-c/4.7.4-intel-20.0.2
-eccodes/2.19.1-intel-20.0.2 netcdf-fortran/4.5.3-gcc-8.3.1
-hdf5/1.10.7-gcc-8.3.1 netcdf-fortran/4.5.3-intel-20.0.2
-hdf5/1.10.7-intel-20.0.2 openmpi/3.1.6-gcc-8.3.1
-intel-parallel-studio/composer.2020.2-intel-20.0.2 openmpi/3.1.6-intel-20.0.2
-```
## Information on SRVX8 - Research
Researchers @UZA2
[Go to Description](SRVX8.md)
-### Architecture
-| Name | Value |
-| --- | --- |
-| Product | PowerEdge R730xd |
-| Distro | CentOS 6.10 Final |
-| Kernel | 2.6.32-754.29.2.el6.centos.plus.x86_64 GNU/Linux |
-| Processor | Intel(R) Xeon(R) CPU E5-2697 v3 @ 2.60GHz |
-| Cores | 2 CPU, 14 physical cores per CPU, total 56 logical CPU units |
-| BaseBoard | Dell Inc. 0599V5 Intel Corporation C610/X99 series chipset |
-| CPU time | 245 kh |
-| Memory | 504 GB Total |
-| Memory/Core | 18 Gb |
-| Network | 10 Gbit/s |
-### Software
- - GNU compilers, OpenMPI, HDF5, NetCDF, Eccodes, ...
- - Anaconda Python
-```bash
------------------ /opt/spack/share/spack/modules/linux-centos6-haswell/ -----------------
-anaconda3/2020.07-gcc-5.3.0 netcdf-c/4.7.4-gcc-5.3.0
-eccodes/2.18.0-gcc-5.3.0 netcdf-fortran/4.5.3-gcc-5.3.0
-gcc/4.4.7-gcc-5.3.0 openmpi/3.1.6-gcc-5.3.0
-gcc/5.3.0-gcc-5.3.0 proj/7.1.0-gcc-5.3.0
-hdf5/1.10.7-gcc-5.3.0 python/3.8.5-gcc-5.3.0
-miniconda3/4.8.2-gcc-5.3.0 zlib/1.2.11-gcc-5.3.0
-```
-
## Information on VSC - Research
Researchers can use the private nodes at VSC-4.
@@ -181,29 +57,5 @@ Note:
> https://vsc.ac.at/training
-
-
[Go to Description](VSC.md)
-### Architecture
-| Name | Value |
-| --- | --- |
-| Product | ? |
-| Distro | CentOS 7.6 |
-| Kernel | 3.10.0-957.el7.x86_64 |
-| Processor | Intel(R) Xeon(R) Gold 6148 CPU @ 2.40GHz |
-| Cores | 2 CPU, 20 physical cores per CPU, total 80 logical CPU units |
-| BaseBoard | ? |
-| CPU time | 2102 kh |
-| Memory | 378 GB Total |
-| Memory/Core | 13.5 Gb |
-| Network | 10 Gbit/s |
-
-### Software
- - GNU compilers, OpenMPI, HDF5, NetCDF, Eccodes (only GNU)
- - Anaconda Python
- - Matlab
-
-
-
-
diff --git a/SRVX1.md b/SRVX1.md
index 9f02d89..6e04b08 100644
--- a/SRVX1.md
+++ b/SRVX1.md
@@ -5,37 +5,7 @@
## Getting Started
-### SSH
-- [x] Terminal, Putty, ...
-- [x] Network access @UNIVIE
-
-Use a terminal or [Putty](https://www.putty.org/) to connect with SSH to the server
-```bash
-ssh [user]@srvx1.img.univie.ac.at
-```
-Please replace `[user]` with the username given by the [sysadmin](mailto:michael.blaschek@univie.ac.at) and supply your password when asked for. This should give you access to the login node. If you are outside of the university network, then please goto [VPN](#VPN) below.
-
-### VPN
-- [x] `u:account`
-
-The jet cluster is only accessible from within the virtual network of the university of Vienna. Therefore access from outside has to be granted via the [VPN-Service](https://vpn.univie.ac.at). Go there and login with your `u:account` and download the *Big-IP Edge* client for you system.
-
-
-
-Links:
- * [ZID-VPN](https://vpn.univie.ac.at/f5-w-68747470733a2f2f7a69642e756e697669652e61632e6174$$/vpn/)
- * Linux (Ubuntu, Generic), Windows, Mac: [VPN user guides](https://vpn.univie.ac.at/f5-w-68747470733a2f2f7a69642e756e697669652e61632e6174$$/vpn/anleitungen/)
- * Arch based AUR package [AUR f5fpc](https://aur.archlinux.org/packages/f5fpc/)
-
-Follow the install instructions for Windows, Mac and Linux and make sure the software works.
-
-
-
-On Windows and MAc you get a nice gui that requires you to fill in the VPN server: `vpn.univie.ac.at` and username and password from the `u:account`. On Linux execute the following:
-```
-f5fpc -s -t vpn.univie.ac.at -u [user]
-```
-The status can be checked with `f5fpc --info`.
+[How to SSH / VNC / VPN](SSH-VPN-VNC/README.md)
### Jupyterhub
<img src="https://jupyter.org/assets/hublogo.svg" width="300px">
@@ -52,17 +22,12 @@ Download/Use any of these required Authenticator Apps:
After registering the teacher/admin has to grant you access and you can login.
-:construction:
-
## System Information
| Name | Value |
| --- | --- |
| Product | PowerEdge R720xd |
-| Distro | CentOS 6.10 Final |
-| Kernel | 2.6.32-754.29.2.el6.centos.plus.x86_64 GNU/Linux |
| Processor | Intel(R) Xeon(R) CPU E5-2690 0 @ 2.90GHz |
| Cores | 2 CPU, 8 physical cores per CPU, total 32 logical CPU units |
-| BaseBoard | Dell Inc. 0C4Y3R Intel Corporation C600/X79 series chipset |
| CPU time | 140 kh |
| Memory | 190 GB Total |
| Memory/Core | 11.9 GB |
@@ -70,6 +35,45 @@ After registering the teacher/admin has to grant you access and you can login.
## Software
+The typcial installation of a intel-cluster has the INTEL Compiler suite (`intel-parallel-studio`) and the open source GNU Compilers installed. Based on these two different compilers (`intel`, `gnu`), there are usually two version of each scientific software.
+Major Libraries:
+ - OpenMPI (3.1.6, 4.0.5)
+ - HDF5
+ - NetCDF (C, Fortran)
+ - ECCODES from [ECMWF](https://confluence.ecmwf.int/display/ECC)
+ - Math libraries e.g. intel-mkl, lapack,scalapack
+ - Interpreters: Python, Julia
+ - Tools: cdo, ncl, nco, ncview
+
+These software libraries are usually handled by environment modules.
+
+<img src="http://modules.sourceforge.net/modules_red.svg" width="300px">
+
+## Currently installed modules
+
+```
+$ module av
+-------------------------------- /home/opt/spack/share/spack/modules/linux-centos6-sandybridge --------------------------------
+anaconda3/2020.07-gcc-5.3.0 gcc/5.3.0-gcc-5.3.0 netcdf-c/4.7.4-gcc-5.3.0 zlib/1.2.11-gcc-5.3.0
+cdo/1.9.9-gcc-5.3.0 git/2.29.0-gcc-5.3.0 netcdf-fortran/4.5.3-gcc-5.3.0
+eccodes/2.18.0-gcc-5.3.0 hdf5/1.10.7-gcc-5.3.0 openmpi/3.1.6-gcc-5.3.0
+enstools/2020.11.dev-gcc-5.3.0 miniconda3/4.8.2-gcc-5.3.0 proj/7.1.0-gcc-5.3.0
+```
+Using [environment modules](https://modules.readthedocs.io/en/latest/) it is possible to have different software libraries (versions, compilers) side-by-side and ready to be loaded. Be aware that some libraries are dependent on others. It is recommended to load the highest rank library first to check what dependencies are loaded as well. e.g.:
+```
+$ module load eccodes/2.18.0-gcc-5.3.0
+Loading eccodes/2.18.0-gcc-5.3.0
+ Loading requirement: zlib/1.2.11-gcc-5.3.0 openmpi/3.1.6-gcc-5.3.0 hdf5/1.10.7-gcc-5.3.0 netcdf-c/4.7.4-gcc-5.3.0
+```
+loads the `ECCODES` library and all dependencies. e.g. intel or gnu compilers, as indicated by the naming.
+```
+$ module list
+Currently Loaded Modulefiles:
+ 1) zlib/1.2.11-gcc-5.3.0 3) hdf5/1.10.7-gcc-5.3.0 5) eccodes/2.18.0-gcc-5.3.0
+ 2) openmpi/3.1.6-gcc-5.3.0 4) netcdf-c/4.7.4-gcc-5.3.0
+```
+`module list` shows the currently loaded modules and reports that 5 libraries need to be loaded as dependencies for `ECCODES`. Thus, it is not necessary to load the other libraries manually as they are dependencies of `ECCODES`.
+
## Best Practice
## Networking
diff --git a/SRVX2.md b/SRVX2.md
index 6300fba..d6790e9 100644
--- a/SRVX2.md
+++ b/SRVX2.md
@@ -1,31 +1,36 @@
-# S R V X 8
+# S R V X 2
[[_TOC_]]
## Getting Started
-### SSH
-
-### VPN
-
-### Jupyterhub
+[How to SSH / VNC / VPN](SSH-VPN-VNC/README.md)
## System Information
| Name | Value |
| --- | --- |
| Product | PowerEdge R940 |
-| Distro | CentOS 8.2.2004 Core |
-| Kernel | 4.18.0-147.el8.x86_64 GNU/Linux |
| Processor | Intel(R) Xeon(R) Gold 6148 CPU @ 2.40GHz |
| Cores | 4 CPU, 20 physical cores per CPU, total 160 logical CPU units |
-| BaseBoard | Dell Inc. 0D41HC Intel Corporation C621 Series Chipset |
| CPU time | 700 kh |
| Memory | 376 GB Total |
| Memory/Core | 9.4 GB |
## Software
+```
+$ module av
+------------------------------ /home/spack-root/share/spack/modules/linux-centos8-skylake_avx512 ------------------------------
+anaconda3/2020.07-gcc-8.3.1 intel-parallel-studio/composer.2020.2-intel-20.0.2 netcdf-fortran/4.5.3-gcc-8.3.1
+eccodes/2.18.0-intel-20.0.2 libemos/4.5.9-gcc-8.3.1 netcdf-fortran/4.5.3-intel-20.0.2
+eccodes/2.19.1-gcc-8.3.1 miniconda3/4.8.2-gcc-8.3.1 netlib-lapack/3.8.0-gcc-8.3.1
+eccodes/2.19.1-intel-20.0.2 miniconda3/4.9.2-gcc-8.3.1 netlib-scalapack/2.1.0-gcc-8.3.1
+eccodes/2.21.0-intel-20.0.2 ncl/6.6.2-gcc-8.3.1 openblas/0.3.12-gcc-8.3.1
+hdf5/1.10.7-gcc-8.3.1 netcdf-c/4.7.4-gcc-8.3.1 openmpi/3.1.6-gcc-8.3.1
+hdf5/1.10.7-intel-20.0.2 netcdf-c/4.7.4-intel-20.0.2 openmpi/3.1.6-intel-20.0.2
+```
+
## Best Practice
## Networking
diff --git a/SRVX8.md b/SRVX8.md
index ce99ce1..3847ead 100644
--- a/SRVX8.md
+++ b/SRVX8.md
@@ -5,22 +5,15 @@
## Getting Started
-### SSH
-
-### VPN
-
-### Jupyterhub
+[How to SSH / VNC / VPN](SSH-VPN-VNC/README.md)
## System Information
| Name | Value |
| --- | --- |
| Product | PowerEdge R730xd |
-| Distro | CentOS 6.10 Final |
-| Kernel | 2.6.32-754.29.2.el6.centos.plus.x86_64 GNU/Linux |
| Processor | Intel(R) Xeon(R) CPU E5-2697 v3 @ 2.60GHz |
| Cores | 2 CPU, 14 physical cores per CPU, total 56 logical CPU units |
-| BaseBoard | Dell Inc. 0599V5 Intel Corporation C610/X99 series chipset |
| CPU time | 245 kh |
| Memory | 504 GB Total |
| Memory/Core | 18 Gb |
@@ -28,6 +21,9 @@
## Software
+Software is installed in numerous places. This is a legency system with no software controller.
+A reinstall is planed for summer 2021.
+
## Best Practice
## Networking
diff --git a/SSH-VPN-VNC/README.md b/SSH-VPN-VNC/README.md
index 24416ba..f4a8655 100644
--- a/SSH-VPN-VNC/README.md
+++ b/SSH-VPN-VNC/README.md
@@ -46,6 +46,8 @@ Host login
```
and replacing `[USERNAME]` and `[U:Account USERNAME]` with your usernames. Using such a file allows to connect like this `ssh srvx2` using the correct server adress and specified username. Copy this file as well on `login.univie.ac.at` and you can use commands like this: `ssh -t login ssh jet` to connect directly to `jet` via the `login` gateway.
+If you want to use ssh-keys you can also use different keys in `.ssh/config` per server with `IdentityFile ~/.ssh/id_rsa_for_server`.
+
**From eduroam**: You should be able to log in as above.
@@ -166,6 +168,28 @@ Adding resolutions according to your display's resolution have a look here: [add
Note: `$DISPLAY` is an environment variable that is usually set to your VNC server port.
+# VPN
+Requirements:
+- `u:account`
+
+Some servers are only accessible from within the virtual network of the university of Vienna. Therefore access from outside has to be granted via the [VPN-Service](https://vpn.univie.ac.at). Go there and login with your `u:account` and download the *Big-IP Edge* client for you system.
+
+
+
+Links:
+ * [ZID-VPN](https://vpn.univie.ac.at/f5-w-68747470733a2f2f7a69642e756e697669652e61632e6174$$/vpn/)
+ * Linux (Ubuntu, Generic), Windows, Mac: [VPN user guides](https://vpn.univie.ac.at/f5-w-68747470733a2f2f7a69642e756e697669652e61632e6174$$/vpn/anleitungen/)
+ * Arch based AUR package [AUR f5fpc](https://aur.archlinux.org/packages/f5fpc/)
+
+Follow the install instructions for Windows, Mac and Linux and make sure the software works.
+
+
+
+On Windows and Mac you get a nice gui that requires you to fill in the VPN server: `vpn.univie.ac.at` and username and password from the `u:account`. On Linux execute the following:
+```
+f5fpc -s -t vpn.univie.ac.at -u [user]
+```
+The status can be checked with `f5fpc --info`.
## Screen
[Screen](https://wiki.ubuntuusers.de/Screen/) is terminal session manager, that allows to start processes and reconnect to these processes after disconnection.
--
GitLab