diff --git a/ECMWF/README.md b/ECMWF/README.md index bfa137101f08f70277825afa41abd19bb08b5f11..f03e68d9996df640605a0a8f0aec4cbbe511f87c 100644 --- a/ECMWF/README.md +++ b/ECMWF/README.md @@ -1,6 +1,6 @@ # European Center for Medium-Range Weather Forecast -<img src="../mkdocs/img/logo_ecmwf.png" width="400px"> +{: width="400px"} [website](https://www.ecmwf.int) / [service status](https://www.ecmwf.int/en/service-status) / [confluence](https://confluence.ecmwf.int) / [support](https://support.ecmwf.int) / [accounting](https://www.ecmwf.int/user) diff --git a/Editors/README.md b/Editors/README.md index 3302b1d93f7546b39cab495d09d785fe658b1949..20621520bb22ce2da805fea2c4999f40dcfda21c 100644 --- a/Editors/README.md +++ b/Editors/README.md @@ -8,7 +8,7 @@ Thanks. :elephant: ## Vim - +{: width="100px"} Vim (a contraction of Vi IMproved) is a clone, with additions, of Bill Joy's vi text editor program for Unix. @@ -50,7 +50,7 @@ zc - close folded text ``` ## Emacs - +{: width="100px"} This is another major editor that is available on all unix (linux) machines as well. It has a commandline interface and a gui. It is much easier than VIM, but again has a lot of [shortcuts](Emacs-Reference.pdf) to learn. @@ -62,7 +62,7 @@ There are much more things that can be added to `~/.emacs` if you want. Have a l ## IDEs ### Atom - +{: width="100px"} Atom is a free and open-source text and source code editor for macOS, Linux, and Microsoft Windows with support for plug-ins written in JavaScript, and embedded Git Control, developed by GitHub. Atom is a desktop application built using web technologies. @@ -71,7 +71,7 @@ Install: [Atom](https://atom.io/) This editor can do everything, as it is open source and solutions on common problems exist on github. However, it is for more advanced users, as sometimes you need to change things and fix small bugs. ### VS-Code - +{: width="100px"} Visual Studio Code is a freeware source-code editor made by Microsoft for Windows, Linux and macOS. Features include support for debugging, syntax highlighting, intelligent code completion, snippets, code refactoring, and embedded Git. Microsoft has released Visual Studio Code's source code on the `microsoft/vscode` (Code - OSS) repository of GitHub, under the permissive MIT License, while the releases by Microsoft are freeware. @@ -81,7 +81,7 @@ Open-Source: [OSS](https://github.com/microsoft/vscode) This is a great editor and has some nice features for Python, however PyCharm is much better for Python. A great feature is remote development. Using VS-code with remote development on SRVX1 or SRVX8 can be done using [instructions](vscode.md) and using a [singularity](https://sylabs.io/guides/3.7/user-guide/) container. ### Pycharm - +{: width="100px"} PyCharm is an integrated development environment (IDE) used in computer programming, specifically for the Python language. It is developed by the Czech company JetBrains. It provides code analysis, a graphical debugger, an integrated unit tester, integration with version control systems (VCSes), and supports web development with Django as well as data science with Anaconda. @@ -92,14 +92,14 @@ Install: [Pycharm Community](https://www.jetbrains.com/de-de/pycharm/) This is perfect for python development. However, remote development is not supported by the community version and running it via VNC works. ## Meld - +{: width="100px"} Download: [Meld](https://meldmerge.org/) Meld is a visual diff and merge tool targeted at developers. Meld helps you compare files, directories, and version controlled projects. It provides **two- and three-way** comparison of both files and directories, and has support for many popular version control systems. ## Joplin - +{: width="100px"} Download: [Joplin](https://joplinapp.org) diff --git a/Editors/vscode.md b/Editors/vscode.md index e682829de5bacefa93b5b839fda018cfff57951c..840c0e6bc2b73954d1f4bc315d720a46551092ec 100644 --- a/Editors/vscode.md +++ b/Editors/vscode.md @@ -7,16 +7,31 @@ ## Remote SSH There is package called `Remote Development` in VSCode which includes + - Remote SSH - Remote WSL - Remote Containers +- Remote Explorer + +Although all might be useful, we try to make use of the first one Remote-SSH. This allows to connect with your locally installed VSCode to connect to a remote environment, offering a shell and access to the file system on that host. + + + +Information on: + +- [Remote Development with SSH](https://code.visualstudio.com/docs/remote/ssh) +- [Remote Development with SSH and JumpHost](https://support.cs.wwu.edu/home/survival_guide/tools/VSCode_Jump.html) -Although all might be useful, we try to make use of the first one Remote-SSH. ### Kernel version is too old? -We are going to use a [singularity container](https://sylabs.io/guides/3.7/user-guide/) to run a more recent version of `node` and allow VSCode to use `srvx8` as well. It will complain a bit, but work. + +???+ bug "Update 2023" + Not relevant anymore. All our servers have been upgraded. + +What happens if the server has a too old kernel. We are going to use a [singularity container](https://sylabs.io/guides/3.7/user-guide/) to run a more recent version of `node` and allow VSCode to use `srvx8` as well. It will complain a bit, but work. Here is how: + 1. Login to `srvx8` 2. run `/opt/containers/vscode-server/setup.sh` 3. Look inside `$HOME/.vscode-server/singularity.bash` and make sure that path that you require are available in `SINGULARITY_BIND` variable. diff --git a/Latex/README.md b/Latex/README.md index e952bab7abdaff31b7bdea16caefb21d3bcf1a59..b848892b9727e621be7ae72874df71f6f956b00e 100644 --- a/Latex/README.md +++ b/Latex/README.md @@ -66,5 +66,5 @@ Links: - [Herbert Voß - Latex Referenz der Umgebungen (Makros, Längen, Zähler)](http://www.lehmanns.de/pdf/latexreferenz.pdf) - [Table Converter Online (csv, xml, json, excel, md, tex, ...)](https://tableconvert.com) - [Pandoc Demos](https://pandoc.org/demos.html) -- [Academic Pandoc template](https://maehr.github.io/academic-pandoc-template/) <img src="https://maehr.github.io/academic-pandoc-template/android-chrome-512x512.png" width="30px"> -- [Eisvogel Pandoc Template](https://github.com/Wandmalfarbe/pandoc-latex-template) <img src="https://github.com/Wandmalfarbe/pandoc-latex-template/blob/master/icon.png?raw=true" width="30px"> +- [Academic Pandoc template](https://maehr.github.io/academic-pandoc-template/) {: width="30px"} +- [Eisvogel Pandoc Template](https://github.com/Wandmalfarbe/pandoc-latex-template) {: width="30px"} diff --git a/README.md b/README.md index 1ed403516ecd5abb6c24ec45bc8de8244f7aee43..12d7888250925baf0cf561901821d5bae3f1bbb1 100644 --- a/README.md +++ b/README.md @@ -1,5 +1,5 @@ -<img src="./mkdocs/img/logo_uniwien.jpg" width="300px"> -<img src="./mkdocs/img/logo_img2_color.png" width="300px"> +{: width="300px"} +{: width="300px"} # Welcome to the Department of Meteorology and Geophysics diff --git a/SSH-VPN-VNC/Questions.md b/SSH-VPN-VNC/Questions.md index 8e58c2f301a37364ddae07a751f7deeef9ffed5d..0df7b5e84c943e7941c8e568acf7b7df069b7577 100644 --- a/SSH-VPN-VNC/Questions.md +++ b/SSH-VPN-VNC/Questions.md @@ -135,7 +135,7 @@ $ ssh -oForwardAgent=yes srvx1 Keep in mind that you can use the ssh-agent with [KeepassXC](https://keepassxc.org/docs/#faq-ssh-agent-how), find a nice tutorial [here](https://ferrario.me/using-keepassxc-to-manage-ssh-keys/). This is really convenient as it allows you to use all keys in the Keepass as long as it is unlocked. The keys will be automatically removed when the keepass is locked. :) -<img src="https://upload.wikimedia.org/wikipedia/commons/c/c1/KeePassXC.svg" width="200px"> +[{: width="200px"}](https://keepassxc.org/) ## Q: How to transfer files between two VPN networks? diff --git a/Servers/JET.md b/Servers/JET.md index fabcf800364a78b4286e0df0f2bc2305c9b419ad..8cce62d00a9b19b2043c2d4c27d5ad963b42b57a 100644 --- a/Servers/JET.md +++ b/Servers/JET.md @@ -1,9 +1,9 @@ +{: width="100px" align=right} # J E T > Research Cluster for Staff > :rocket: 🔒 -<img src="../mkdocs/img/rocket.png" width="100px"> ## Getting Started @@ -26,12 +26,13 @@ Last Update: 16.07.2024 Node Setup: - 2x Login Nodes (jet01, jet02) - - 17x Compute Nodes (jet03-19) + - 7x Compute Nodes INTEL (jet03-09) + - 10x Compute Nodes AMD (jet10-jet19) - 5x Storage Nodes  -### Example of one Compute Node +### Example INTEL Node | Type | Detail | | --- | --- | @@ -43,12 +44,27 @@ Node Setup: | Memory/Core | 18.9 GB | | Network | 100 Gbit/s (Infiniband) | +### Example AMD Node + +| Type | Detail | +| --- | --- | +| Product | ThinkSystem SR635 V3 | +| Processor | AMD EPYC 9454P 48-Core Processor | +| Cores | 1 CPU, 48 physical cores per CPU, total 96 logical CPU units | +| CPU Time | 420 kh | +| Memory | 1132 GB Total | +| Memory/Core | 23.5 GB | +| Network | 200 Gbit/s (Infiniband) | + +### Storage + All nodes are connected to a global file system (GPFS) with about 3.5 PB (~3500 TB) of storage. There is no need to copy files to the compute nodes, your HOME and SCRATCH directories will be available under the same path as on the login nodes. Paths: - `/jetfs/home/[username]` - `/jetfs/scratch/[username]` + - `/jetfs/shared-data` ## Software @@ -153,7 +169,7 @@ on how to use environment modules go to [Using Environment Modules](../Misc/Envi ## Jupyterhub -<img src="../mkdocs/img/jupyterhub-logo.svg" width="100px"> +{: width="100px"} The Jet Cluster serves a [jupyterhub](https://jupyterhub.readthedocs.io/en/stable/) with a [jupyterlab](https://jupyterlab.readthedocs.io/en/stable/) that launches on the JET cluster compute nodes and allows users to work directly on the cluster as well as submit jobs. @@ -166,11 +182,11 @@ Steps: **Please use the resources responsible. We trust that you apply a fair-share policy and collaborate with your colleagues.** -<img src="../mkdocs/img/jet-login.png" width="500px"> +{: width="500px"} -<img src="../mkdocs/img/jet-job1.png" width="500px"> +{: width="500px"} -<img src="../mkdocs/img/jet-job2.png" width="500px"> +{: width="500px"} There are several kernels available as modules and how to use other kernels can be found here: @@ -227,13 +243,20 @@ How to use a vnc server, go to [VNC](../SSH-VPN-VNC/VNC.md). ## Network drives -Transfer of files between SRVX1 and JET is **not** necessary. The file system from SRVX1 is mounted on the JET cluster and vice versa. These mounted drives need to transfer the data via the network and latencies might be higher. However, since there is a strong connection between SRVX1 and JET, it might be applicable for you to just read the data over the network. +Transfer of files between SRV and JET is **not** necessary. The file system is mounted on JET Nodes JET01/JET02 and vice versa. These mounted drives need to transfer the data via the network and latencies might be higher. ```bash title="Mounted files systems" $ df -h -131.130.157.11:/mnt/users 319T 250T 70T 79% /mnt/users -131.130.157.11:/mnt/scratch 400T 350T 51T 88% /mnt/scratch -remjetfs 3.6P 1.6P 2.0P 44% /jetfs + +131.130.157.5:/mnt/users 319T 300T 20T 95% /mnt/users +131.130.157.5:/mnt/scratch 400T 321T 80T 81% /mnt/scratch +131.130.157.5:/mnt/users/staff 319T 300T 20T 95% /srvfs/home +131.130.157.5:/mnt/users/scratch 319T 300T 20T 95% /srvfs/tmp +131.130.157.5:/mnt/users/data 319T 300T 20T 95% /srvfs/data +131.130.157.5:/mnt/scratch/scratch 400T 321T 80T 81% /srvfs/scratch +131.130.157.5:/mnt/scratch/shared 400T 321T 80T 81% /srvfs/shared +131.130.157.5:/mnt/scratch/webdata 400T 321T 80T 81% /srvfs/webdata +remjetfs 3.6P 1.6P 2.0P 44% /jetfs ``` ## Slurm diff --git a/Students/TeachingHub.md b/Students/TeachingHub.md index fb83eaa6c6c599fd0ca81e7bc28d5c8a7fb57e33..101116d58a56ce1f12355a6b141514754d14a1e2 100644 --- a/Students/TeachingHub.md +++ b/Students/TeachingHub.md @@ -1,12 +1,13 @@ # Jupyter Notebook Lab -<img src="../mkdocs/img/jupyter-logo.png" width="300px"> +{: width="300px"} Please notice that only the new teaching hub is available in the future. -Migration for the old teaching hub will finish at the end of WS2023/24. +Migration for the old teaching hub will finish at the end of WS2023/24. + Access: [teaching.wolke.img.univie.ac.at](https://teaching.wolke.img.univie.ac.at) or via your [Moodle lecture](https://moodle.univie.ac.at/my/). -<img align="right" width="150px" src="../mkdocs/img/Masterhub_logo.png"> +{: width="200px" align=right} ## MasterHub @@ -16,9 +17,9 @@ Students need to get a server account for accessing this resource. However, it i ## TeachingHub -<img width="200px" src="../mkdocs/img/moodlelogo_small.png"> +{: width="200px"} -???+ warning "Purge" +???+ warning "Annual purge of HOME" It is necessary for a smooth operation to purge the contents of the TeachingHub on an annual basis. Every year before the winter semester starts there will be a purge of students home directories. What is available? diff --git a/mkdocs.yml b/mkdocs.yml index c459f7b831e83f013d8e13ad7adf97d2fcb8efa2..a029de2226c8115d67892c66d9fffdfcc5ee52a4 100644 --- a/mkdocs.yml +++ b/mkdocs.yml @@ -79,3 +79,4 @@ markdown_extensions: - pymdownx.details - pymdownx.emoji - mkdocs_graphviz + - attr_list diff --git a/mkdocs/img/logo-keepassxc.png b/mkdocs/img/logo-keepassxc.png new file mode 100644 index 0000000000000000000000000000000000000000..480683514d8eb174fcbd807bbbf1cf2aad0fcfb1 Binary files /dev/null and b/mkdocs/img/logo-keepassxc.png differ diff --git a/mkdocs/vids/vscode-remote-ssh.gif b/mkdocs/vids/vscode-remote-ssh.gif new file mode 100644 index 0000000000000000000000000000000000000000..3fdd0f9bc6fe7f0a546b54a8eb63c89edfe87f83 Binary files /dev/null and b/mkdocs/vids/vscode-remote-ssh.gif differ