diff --git a/Fortran/README.md b/Fortran/README.md new file mode 100644 index 0000000000000000000000000000000000000000..c0fea9892e6be4a76b8fbd55820583cb21db922b --- /dev/null +++ b/Fortran/README.md @@ -0,0 +1,74 @@ +# Fortran + +Fortran is quite popular in Meteorology and Geophysics. +Please find some help on solving common problems. + +Get some information on Fortran: +- [Fortran Language, Learning, Compilers](https://fortran-lang.org) + + + + +# Compilers + +There are a few compilers, but most commonly GNU (Gfortran) and INTEL (ifort) are used on our servers. + +| | gfortran | ifort | +|----------------------------------|------------------------------------|-----------------------------------| +| | gfortran | ifort | +| double precision real | -fdefault-real-8 | -r8 | +| check array bounds | -fbounds-check | -check | +| call chain traceback | -fbacktrace | -traceback | +| convert little/big endian | -fconvert=big-endian/little-endian | -convert big_endian/little_endian | +| default optimisation | -O0 | -O2 | +| highest recommended optimisation | -O3 | -O2maybe -O3 or -fast | + +## Intel Compiler + +from P. Seibert using ifort for the fastest code (srvx1): +```lang-mk +FFLAGS = -cpp -xAVX -ipo -O3 -no-prec-div -opt-prefetch -m64 -mcmodel=medium -I$(INCPATH) +LIBPATH = /home/tmc/TestEnv/Libraries/grib_api-1.12.3_ifort/lib +LDFLAGS = $(FFLAGS) -L$(LIBPATH) -Bstatic -lgrib_api_f90 -lgrib_api -lm -ljasper +``` + +Remark: for FLEXPART, otherwise you won't need `grib_api` and `jasper`, or `mcmodel=medium`) + +Remarks +these are settings for FLEXPART +in general, you probably won't need the gribapi and jasper library + -cpp -mcmodel=medium -I$(INCPATH) +-lm includes the intel math lib; it is improtant that the linking step is something like $(FC) $(FFLAGS) *.o -o a.out $(LDFLAGS) or you may loose the math lib + +## Tricky Issues + +### record markers +On 64-bit machines, some Fortran compilers will insert record markers that are 64-bit integers instead of the standard 32-bit integers. + +gfortran man page says: +``` +-frecord-marker=length +Specify the length of record markers for unformatted files. Valid values for length are 4 and 8. Default is 4. This is different from previous versions of gfortran, which specified a default record marker length of 8 on most systems. If you want to read or write files compatible with earlier versions of gfortran, use -frecord-marker=8. +``` + +# Trainings + +online training courses ? + + +# Code Testing +pFUnit - python Parallel Fortran Unit Testing Framework [GitHub](https://github.com/Goddard-Fortran-Ecosystem/pFUnit) + + +# Documentation + +## Automatic Documentation Generation +FORD: [Overview in Fortran Wiki](http://fortranwiki.org/fortran/show/FORD) | [Code on Github](https://github.com/cmacmackin/ford) + +## Documentation + +- Fortran wikibook [DE](https://de.wikibooks.org/wiki/Fortran) [EN](https://en.wikibooks.org/wiki/Fortran) + +### Books + + diff --git a/Jet-Cluster.md b/Jet-Cluster.md index 91b28af9bf4e32e1f8e5c055cf8dab12a10e1dd4..012a27cf4f94db995664d4810ba1eaa437116fc7 100644 --- a/Jet-Cluster.md +++ b/Jet-Cluster.md @@ -119,19 +119,8 @@ miniconda3/4.8.2-gcc-8.3.1-3m7b6t2 zlib/1.2.11-gcc-8.3. ncl/6.6.2-gcc-8.3.1-MPI3.1.6-3dxuv5f zlib/1.2.11-intel-20.0.2-3h374ov nco/4.9.3-gcc-8.3.1-g7o6lao ``` +on how to use environment modules go to [Using Environment Modules](Misc/Environment-Modules.md) -Using [environment modules](https://modules.readthedocs.io/en/latest/) it is possible to have different software libraries (versions, compilers) side-by-side and ready to be loaded. Be aware that some libraries are dependent on others. It is recommended to load the highest rank library first to check what dependencies are loaded as well. e.g.: -``` -$ module load eccodes/2.18.0-intel-20.0.2-6tadpgr -``` -loads the `ECCODES` library and all dependencies. e.g. intel compilers, as indicated by the naming. -``` -$ module list -Currently Loaded Modulefiles: - 1) zlib/1.2.11-intel-20.0.2-3h374ov 3) hdf5/1.12.0-intel-20.0.2-ezeotzr 5) netcdf-c/4.7.4-intel-20.0.2-337uqtc - 2) openmpi/4.0.5-intel-20.0.2-4wfaaz4 4) parallel-netcdf/1.12.1-intel-20.0.2-sgz3yqs 6) eccodes/2.18.0-intel-20.0.2-6tadpgr -``` -`module list` shows the currently loaded modules and reports that 6 libraries need to be loaded as dependencies for `ECCODES`. Thus, it is not necessary to load the other libraries manually as they are dependencies of `ECCODES`. ## Slurm [Slurm Tutorial on Gitlab](https://gitlab.phaidra.org/imgw/slurm) 🔒 diff --git a/Misc/Environment-Modules.md b/Misc/Environment-Modules.md new file mode 100644 index 0000000000000000000000000000000000000000..52677eb1e5afd84e85cd80116d6418bbc09bdee3 --- /dev/null +++ b/Misc/Environment-Modules.md @@ -0,0 +1,131 @@ +# Using environment modules + + + +Using [environment modules](https://modules.readthedocs.io/en/latest/) it is possible to have different software libraries (versions, compilers) side-by-side and ready to be loaded. Be aware that some libraries are dependent on others. It is recommended to load the highest rank library first to check what dependencies are loaded as well. e.g.: +```bash +$ module load eccodes/2.18.0-intel-20.0.2-6tadpgr +``` +loads the `ECCODES` library and all dependencies. e.g. intel compilers, as indicated by the naming. +```bash +$ module list +Currently Loaded Modulefiles: + 1) zlib/1.2.11-intel-20.0.2-3h374ov 3) hdf5/1.12.0-intel-20.0.2-ezeotzr 5) netcdf-c/4.7.4-intel-20.0.2-337uqtc + 2) openmpi/4.0.5-intel-20.0.2-4wfaaz4 4) parallel-netcdf/1.12.1-intel-20.0.2-sgz3yqs 6) eccodes/2.18.0-intel-20.0.2-6tadpgr +``` +`module list` shows the currently loaded modules and reports that 6 libraries need to be loaded as dependencies for `ECCODES`. Thus, it is not necessary to load the other libraries manually as they are dependencies of `ECCODES`. However it will be necessary to load the intel compiler suite `intel-parallel-studio/composer.2020.2-intel-20.0.2-zuot22y` as well for build applications. + + +```bash +# unload modules +$ module unload eccodes/2.18.0-intel-20.0.2-6tadpgr + +# unload all modules at once (useful in jobs, before loading the correct ones) +$ module purge + +# show information from a module (defined variables) +$ module show eccodes/2.18.0-intel-20.0.2-6tadpgr +------------------------------------------------------------------- +/jetfs/spack/share/spack/modules/linux-rhel8-skylake_avx512/eccodes/2.18.0-intel-20.0.2-6tadpgr: + +module-whatis ecCodes is a package developed by ECMWF for processing meteorological data in GRIB (1/2), BUFR (3/4) and GTS header formats. +conflict eccodes +prepend-path PATH /jetfs/spack/opt/spack/linux-rhel8-skylake_avx512/intel-20.0.2/eccodes-2.18.0-6tadpgreot7jf4yoaiqmqueiihhdcsxk/bin +prepend-path LIBRARY_PATH /jetfs/spack/opt/spack/linux-rhel8-skylake_avx512/intel-20.0.2/eccodes-2.18.0-6tadpgreot7jf4yoaiqmqueiihhdcsxk/lib +prepend-path LD_LIBRARY_PATH /jetfs/spack/opt/spack/linux-rhel8-skylake_avx512/intel-20.0.2/eccodes-2.18.0-6tadpgreot7jf4yoaiqmqueiihhdcsxk/lib +prepend-path C_INCLUDE_PATH /jetfs/spack/opt/spack/linux-rhel8-skylake_avx512/intel-20.0.2/eccodes-2.18.0-6tadpgreot7jf4yoaiqmqueiihhdcsxk/include +prepend-path CPLUS_INCLUDE_PATH /jetfs/spack/opt/spack/linux-rhel8-skylake_avx512/intel-20.0.2/eccodes-2.18.0-6tadpgreot7jf4yoaiqmqueiihhdcsxk/include +prepend-path INCLUDE /jetfs/spack/opt/spack/linux-rhel8-skylake_avx512/intel-20.0.2/eccodes-2.18.0-6tadpgreot7jf4yoaiqmqueiihhdcsxk/include +prepend-path PKG_CONFIG_PATH /jetfs/spack/opt/spack/linux-rhel8-skylake_avx512/intel-20.0.2/eccodes-2.18.0-6tadpgreot7jf4yoaiqmqueiihhdcsxk/lib/pkgconfig +prepend-path CMAKE_PREFIX_PATH /jetfs/spack/opt/spack/linux-rhel8-skylake_avx512/intel-20.0.2/eccodes-2.18.0-6tadpgreot7jf4yoaiqmqueiihhdcsxk/ +------------------------------------------------------------------- +``` + +## Compilers + +### GNU +gfortran is available on the default command search path, so it is not essential to load a module, unless: (1) You need to set environment variables such as FC and F77; (2) You need to link MPI libraries. In these cases, you should load the gfortran module. This is what the module does to the environment: +``` +$ module show gcc/8.3.1-gcc-8.3.1-pp3wjou +------------------------------------------------------------------- +/jetfs/spack/share/spack/modules/linux-rhel8-skylake_avx512/gcc/8.3.1-gcc-8.3.1-pp3wjou: + +module-whatis The GNU Compiler Collection includes front ends for C, C++, Objective-C, Fortran, Ada, and Go, as well as libraries for these languages. +conflict gcc +prepend-path MANPATH /usr/share/man +prepend-path ACLOCAL_PATH /usr/share/aclocal +prepend-path PKG_CONFIG_PATH /usr/lib64/pkgconfig +prepend-path PKG_CONFIG_PATH /usr/share/pkgconfig +setenv CC /usr/bin/gcc +setenv CXX /usr/bin/g++ +setenv FC /usr/bin/gfortran +setenv F77 /usr/bin/gfortran +prepend-path LD_LIBRARY_PATH /usr/lib64:/usr/lib +prepend-path PATH /usr/bin +prepend-path CMAKE_PREFIX_PATH /usr +setenv F90 /usr/bin/gfortran +------------------------------------------------------------------- +``` + +### INTEL +there are a few version of intel compilers installed and some more might be added as well: +```bash +$ module av intel +----------------------- /jetfs/spack/share/spack/modules/linux-rhel8-skylake_avx512 ------------------------ +intel-mkl/2020.3.279-gcc-8.3.1-5xeezjw +intel-mkl/2020.3.279-intel-20.0.2-m7bxged +intel-oneapi-compilers/2021.2.0-oneapi-2021.2.0-6kdzddx +intel-oneapi-mpi/2021.2.0-oneapi-2021.2.0-haqpxfl +intel-parallel-studio/composer.2020.2-intel-20.0.2-zuot22y + +--------------------------- /jetfs/spack/share/spack/modules/linux-rhel8-haswell --------------------------- +intel-parallel-studio/composer.2017.7-intel-17.0.7-disfj2g +``` +This shows that we have `intel-parallel-studio` with version `20.0.2` and `17.0.7` installed and `intel-oneapi-compilers` at version `2021.2.0`. The first does not come with Intel-MPI, but the second does. + +Again the module sets a lot of +```bash +$ module show intel-parallel-studio/composer.2020.2-intel-20.0.2-zuot22y +------------------------------------------------------------------- +/jetfs/spack/share/spack/modules/linux-rhel8-skylake_avx512/intel-parallel-studio/composer.2020.2-intel-20.0.2-zuot22y: + +module-whatis Intel Parallel Studio. +conflict intel-parallel-studio +prepend-path PATH /jetfs/spack/opt/spack/linux-rhel8-skylake_avx512/intel-20.0.2/intel-parallel-studio-composer.2020.2-zuot22yfoe7jl67ttimvkzghwluvyaas/bin +prepend-path MANPATH /jetfs/spack/opt/spack/linux-rhel8-skylake_avx512/intel-20.0.2/intel-parallel-studio-composer.2020.2-zuot22yfoe7jl67ttimvkzghwluvyaas/man +prepend-path LIBRARY_PATH /jetfs/spack/opt/spack/linux-rhel8-skylake_avx512/intel-20.0.2/intel-parallel-studio-composer.2020.2-zuot22yfoe7jl67ttimvkzghwluvyaas/lib +prepend-path LD_LIBRARY_PATH /jetfs/spack/opt/spack/linux-rhel8-skylake_avx512/intel-20.0.2/intel-parallel-studio-composer.2020.2-zuot22yfoe7jl67ttimvkzghwluvyaas/lib +prepend-path C_INCLUDE_PATH /jetfs/spack/opt/spack/linux-rhel8-skylake_avx512/intel-20.0.2/intel-parallel-studio-composer.2020.2-zuot22yfoe7jl67ttimvkzghwluvyaas/include +prepend-path CPLUS_INCLUDE_PATH /jetfs/spack/opt/spack/linux-rhel8-skylake_avx512/intel-20.0.2/intel-parallel-studio-composer.2020.2-zuot22yfoe7jl67ttimvkzghwluvyaas/include +prepend-path INCLUDE /jetfs/spack/opt/spack/linux-rhel8-skylake_avx512/intel-20.0.2/intel-parallel-studio-composer.2020.2-zuot22yfoe7jl67ttimvkzghwluvyaas/include +prepend-path CMAKE_PREFIX_PATH /jetfs/spack/opt/spack/linux-rhel8-skylake_avx512/intel-20.0.2/intel-parallel-studio-composer.2020.2-zuot22yfoe7jl67ttimvkzghwluvyaas/ +prepend-path CLASSPATH ...linux/mpi/intel64/lib/mpi.jar:...linux/daal/lib/daal.jar +prepend-path CPATH ...linux/ipp/include:...linux/mkl/include:...linux/pstl/include:...linux/pstl/stdlib:...linux/tbb/include:...linux/tbb/include:...linux/daal/include +setenv DAALROOT ...linux/daal +prepend-path FI_PROVIDER_PATH ...linux/mpi/intel64/libfabric/lib/prov +prepend-path INTEL_LICENSE_FILE /jetfs/spack/opt/spack/linux-rhel8-skylake_avx512/intel-20.0.2/intel-parallel-studio-composer.2020.2-zuot22yfoe7jl67ttimvkzghwluvyaas/licenses:...linux/licenses:/opt/intel/licenses:/home/spack/intel/licenses +setenv IPPROOT ...linux/ipp +setenv I_MPI_ROOT ...linux/mpi +prepend-path LD_LIBRARY_PATH ...linux/compiler/lib/intel64_lin:...linux/mpi/intel64/libfabric/lib:...linux/mpi/intel64/lib/release:...linux/mpi/intel64/lib:...linux/ipp/lib/intel64:...linux/mkl/lib/intel64_lin:...linux/tbb/lib/intel64/gcc4.8:...linux/daal/lib/intel64_lin:...linux/daal/../tbb/lib/intel64_lin/gcc4.4:...linux/daal/../tbb/lib/intel64_lin/gcc4.8 +prepend-path LIBRARY_PATH ...linux/mpi/intel64/libfabric/lib:...linux/ipp/lib/intel64:...linux/compiler/lib/intel64_lin:...linux/mkl/lib/intel64_lin:...linux/tbb/lib/intel64/gcc4.8:...linux/tbb/lib/intel64/gcc4.8:...linux/daal/lib/intel64_lin:...linux/daal/../tbb/lib/intel64_lin/gcc4.4:...linux/daal/../tbb/lib/intel64_lin/gcc4.8 +setenv MKLROOT ...linux/mkl +prepend-path NLSPATH ...linux/compiler/lib/intel64/locale/%l_%t/%N:...linux/mkl/lib/intel64_lin/locale/%l_%t/%N +prepend-path PKG_CONFIG_PATH ...linux/mkl/bin/pkgconfig +setenv PSTLROOT ...linux/pstl +setenv TBBROOT ...linux/tbb +prepend-path MANPATH /jetfs/spack/opt/spack/linux-rhel8-skylake_avx512/intel-20.0.2/intel-parallel-studio-composer.2020.2-zuot22yfoe7jl67ttimvkzghwluvyaas/man/common::/opt/xcat/share/man:/opt/xcat/share/man::/opt/slurm/share/man:/opt/slurm/share/man: +prepend-path PATH ...linux/bin/intel64:...linux/bin:...linux/mpi/intel64/libfabric/bin:...linux/mpi/intel64/bin:/home/spack/.local/bin:/home/spack/bin:/opt/xcat/bin:/opt/xcat/sbin:/opt/xcat/share/xcat/tools:/jetfs/userservices:/jetfs/home/mblaschek/bin:/jetfs/home/mblaschek/.local/bin:/jetfs/spack/bin:/opt/slurm/bin:/usr/bin:/usr/local/bin:/usr/local/sbin:/usr/sbin:/usr/share/Modules/bin:/jetfs/spack/opt/spack/linux-rhel8-skylake_avx512/intel-20.0.2/intel-parallel-studio-composer.2020.2-zuot22yfoe7jl67ttimvkzghwluvyaas/parallel_studio_xe_2020.2.108/bin +setenv CC /jetfs/spack/opt/spack/linux-rhel8-skylake_avx512/intel-20.0.2/intel-parallel-studio-composer.2020.2-zuot22yfoe7jl67ttimvkzghwluvyaas/bin/icc +setenv CXX /jetfs/spack/opt/spack/linux-rhel8-skylake_avx512/intel-20.0.2/intel-parallel-studio-composer.2020.2-zuot22yfoe7jl67ttimvkzghwluvyaas/bin/icpc +setenv FC /jetfs/spack/opt/spack/linux-rhel8-skylake_avx512/intel-20.0.2/intel-parallel-studio-composer.2020.2-zuot22yfoe7jl67ttimvkzghwluvyaas/bin/ifort +setenv F77 /jetfs/spack/opt/spack/linux-rhel8-skylake_avx512/intel-20.0.2/intel-parallel-studio-composer.2020.2-zuot22yfoe7jl67ttimvkzghwluvyaas/bin/ifort +setenv F90 /jetfs/spack/opt/spack/linux-rhel8-skylake_avx512/intel-20.0.2/intel-parallel-studio-composer.2020.2-zuot22yfoe7jl67ttimvkzghwluvyaas/bin/ifort +setenv CC /jetfs/spack/opt/spack/linux-rhel8-skylake_avx512/intel-20.0.2/intel-parallel-studio-composer.2020.2-zuot22yfoe7jl67ttimvkzghwluvyaas/compilers_and_libraries/linux/bin/intel64/icc +setenv CXX /jetfs/spack/opt/spack/linux-rhel8-skylake_avx512/intel-20.0.2/intel-parallel-studio-composer.2020.2-zuot22yfoe7jl67ttimvkzghwluvyaas/compilers_and_libraries/linux/bin/intel64/icpc +setenv FC /jetfs/spack/opt/spack/linux-rhel8-skylake_avx512/intel-20.0.2/intel-parallel-studio-composer.2020.2-zuot22yfoe7jl67ttimvkzghwluvyaas/compilers_and_libraries/linux/bin/intel64/ifort +setenv F90 /jetfs/spack/opt/spack/linux-rhel8-skylake_avx512/intel-20.0.2/intel-parallel-studio-composer.2020.2-zuot22yfoe7jl67ttimvkzghwluvyaas/compilers_and_libraries/linux/bin/intel64/ifort +setenv F77 /jetfs/spack/opt/spack/linux-rhel8-skylake_avx512/intel-20.0.2/intel-parallel-studio-composer.2020.2-zuot22yfoe7jl67ttimvkzghwluvyaas/compilers_and_libraries/linux/bin/intel64/ifort +prepend-path LD_LIBRARY_PATH /jetfs/spack/opt/spack/linux-rhel8-skylake_avx512/intel-20.0.2/intel-parallel-studio-composer.2020.2-zuot22yfoe7jl67ttimvkzghwluvyaas/lib/intel64 +------------------------------------------------------------------- +``` + diff --git a/SRVX1.md b/SRVX1.md index dc69b15a4e973cb602e59fc09916dc68333f9089..bfb5b231bf669ab3ac470a98918f7b30bc3d798b 100644 --- a/SRVX1.md +++ b/SRVX1.md @@ -59,18 +59,6 @@ cdo/1.9.9-gcc-5.3.0 git/2.29.0-gcc-5.3.0 netcdf-fortran/4.5.3 eccodes/2.18.0-gcc-5.3.0 hdf5/1.10.7-gcc-5.3.0 openmpi/3.1.6-gcc-5.3.0 enstools/2020.11.dev-gcc-5.3.0 miniconda3/4.8.2-gcc-5.3.0 proj/7.1.0-gcc-5.3.0 ``` -Using [environment modules](https://modules.readthedocs.io/en/latest/) it is possible to have different software libraries (versions, compilers) side-by-side and ready to be loaded. Be aware that some libraries are dependent on others. It is recommended to load the highest rank library first to check what dependencies are loaded as well. e.g.: -``` -$ module load eccodes/2.18.0-gcc-5.3.0 -Loading eccodes/2.18.0-gcc-5.3.0 - Loading requirement: zlib/1.2.11-gcc-5.3.0 openmpi/3.1.6-gcc-5.3.0 hdf5/1.10.7-gcc-5.3.0 netcdf-c/4.7.4-gcc-5.3.0 -``` -loads the `ECCODES` library and all dependencies. e.g. intel or gnu compilers, as indicated by the naming. -``` -$ module list -Currently Loaded Modulefiles: - 1) zlib/1.2.11-gcc-5.3.0 3) hdf5/1.10.7-gcc-5.3.0 5) eccodes/2.18.0-gcc-5.3.0 - 2) openmpi/3.1.6-gcc-5.3.0 4) netcdf-c/4.7.4-gcc-5.3.0 -``` -`module list` shows the currently loaded modules and reports that 5 libraries need to be loaded as dependencies for `ECCODES`. Thus, it is not necessary to load the other libraries manually as they are dependencies of `ECCODES`. +on how to use environment modules go to [Using Environment Modules](Misc/Environment-Modules.md) + diff --git a/SRVX2.md b/SRVX2.md index f8a7e3184af5999bf08e294323e23068424a09de..ec8ca05ace874d5488df3d080d1522de292e1a4d 100644 --- a/SRVX2.md +++ b/SRVX2.md @@ -45,18 +45,6 @@ eccodes/2.21.0-intel-20.0.2 ncl/6.6.2-gcc-8.3.1 hdf5/1.10.7-gcc-8.3.1 netcdf-c/4.7.4-gcc-8.3.1 openmpi/3.1.6-gcc-8.3.1 hdf5/1.10.7-intel-20.0.2 netcdf-c/4.7.4-intel-20.0.2 openmpi/3.1.6-intel-20.0.2 ``` -Using [environment modules](https://modules.readthedocs.io/en/latest/) it is possible to have different software libraries (versions, compilers) side-by-side and ready to be loaded. Be aware that some libraries are dependent on others. It is recommended to load the highest rank library first to check what dependencies are loaded as well. e.g.: -``` -$ module load eccodes/2.18.0-gcc-5.3.0 -Loading eccodes/2.18.0-gcc-5.3.0 - Loading requirement: zlib/1.2.11-gcc-5.3.0 openmpi/3.1.6-gcc-5.3.0 hdf5/1.10.7-gcc-5.3.0 netcdf-c/4.7.4-gcc-5.3.0 -``` -loads the `ECCODES` library and all dependencies. e.g. intel or gnu compilers, as indicated by the naming. -``` -$ module list -Currently Loaded Modulefiles: - 1) zlib/1.2.11-gcc-5.3.0 3) hdf5/1.10.7-gcc-5.3.0 5) eccodes/2.18.0-gcc-5.3.0 - 2) openmpi/3.1.6-gcc-5.3.0 4) netcdf-c/4.7.4-gcc-5.3.0 -``` -`module list` shows the currently loaded modules and reports that 5 libraries need to be loaded as dependencies for `ECCODES`. Thus, it is not necessary to load the other libraries manually as they are dependencies of `ECCODES`. +on how to use environment modules go to [Using Environment Modules](Misc/Environment-Modules.md) + diff --git a/SRVX8.md b/SRVX8.md index 9639d988d6a33bba7c9e27828c3a431089df0046..03ecd349fccf75113c12f52c68918bbf3b7e3da9 100644 --- a/SRVX8.md +++ b/SRVX8.md @@ -24,3 +24,4 @@ Software is installed in numerous places. This is a legency system with no software controller. A reinstall is planed for summer 2021. +on how to use environment modules go to [Using Environment Modules](Misc/Environment-Modules.md) diff --git a/SSH-VPN-VNC/Questions.md b/SSH-VPN-VNC/Questions.md index 15e137ac4d8f2eba676a23621d6fcd742f5c7e36..0f84e97f6983b6b148b2fe799fac8e54b933f2b1 100644 --- a/SSH-VPN-VNC/Questions.md +++ b/SSH-VPN-VNC/Questions.md @@ -33,7 +33,7 @@ The key's randomart image is: It is recommended to use a password to encrpyt the private key `.ssh/id_rsa`. However, this password is then required each time to login. Using an `ssh-agent` can solve that problem. ## Q: How to use an ssh-agent? -Using an SSH-Agent will make your connection even safer, as your private key is encrypted with a passphrase. To create a ssh-key goto [How to use ssh-key authentication?](#How-to-use-ssh-key-authentication?). +Using an SSH-Agent will make your connection even safer, as your private key is encrypted with a passphrase. To create a ssh-key goto [How to use ssh-key authentication?](#q-how-to-use-ssh-key-authentication). Continue with how to use the ssh-agent: ```bash diff --git a/SSH-VPN-VNC/SSH.md b/SSH-VPN-VNC/SSH.md index 861f42fdcb9a05908a77dcb3448053ba8ddc42b8..eebadb19397c9d035b0c7ac791be593efa9bad71 100644 --- a/SSH-VPN-VNC/SSH.md +++ b/SSH-VPN-VNC/SSH.md @@ -41,12 +41,15 @@ If you want to use ssh-keys you can also use different keys in `.ssh/config` per **From eduroam**: You should be able to log in as above. -**From the outer world**: First connect to the [UniVie VPN](https://zid.univie.ac.at/en/vpn/), then start up a terminal and log in as above. Consider using [connect2vpn](connect2vpn) script to do so. Set your U:Account username as environmental variable `$VPN_USER` and run `$ connect2vpn`, select split (only adresses within the UNINET) or full (all connections) tunnel and watch the connection information update every 10 seconds. +**From the outer world**: First connect to the [UniVie VPN](https://zid.univie.ac.at/en/vpn/), then start up a terminal and log in as above. Consider using [connect2vpn](connect2vpn) script to do so. Set your U:Account username as environmental variable `$VPN_USER` and run `connect2vpn`, select split (only adresses within the UNINET) or full (all connections) tunnel and watch the connection information update every 10 seconds. If you are a guest, you can apply for a [guest u:account](https://zid.univie.ac.at/uaccount/#c11096). This will give you access to eduroam and to the VPN. Your application needs to be endorsed by a staff member, who also determines the expiration date of the account. +## SSH Authentication with keys -### Connect Script +Find a solution [Questions - How to use ssh-key authentication?](Questions.md#q-how-to-use-ssh-key-authentication) or [Questions - How to use an ssh-agent?](Questions.md#q-how-to-use-an-ssh-agent) + +## Connect Script If you are using a terminal (Mac, Linux, WSL, ...) you can use the script [connect2jet](connect2jet) like this: ```bash connect2jet -g [U:Account-Username]@login.univie.ac.at [Jet-Username]@jet01.img.univie.ac.at @@ -58,3 +61,32 @@ There is also an option to forward a port, e.g. the VNC Port: connect2jet -g [U:Account-Username]@login.univie.ac.at -p 5901 [Jet-Username]@jet01.img.univie.ac.at ``` which allows you to connect to `localhost:5901` and view the VNC session. Other gateway servers can be `srvx1.img.univie.ac.at` + +## Tunneling + +If you are connected to eduroam or you are on an external computer, you'll need to use an SSH tunnel. The instructions below refer to jet01, but you can do just the same with jet02. + +On Linux, start [Remmina](https://remmina.org/), then: + +* Set "Server" to `jet01.img.univie.ac.at:[DISPLAY]` in the "Basic" tab +* Move to the "SSH Tunnel" tab, checkout "Enable SSH Tunnel", "Same server at port 22" and specify your favourite SSH authentication method. +* Save and connect. + +On Windows, you can use either [SSVNC](http://www.karlrunge.com/x11vnc/ssvnc.html), or the combination of [Bitvise SSH Client](https://www.bitvise.com/ssh-client-download) (for the SSH tunnel) and the [RealVNC VNC Viewer](https://www.realvnc.com/en/connect/download/viewer/windows/). + +Option 1: [SSVNC](http://www.karlrunge.com/x11vnc/ssvnc.html) + +* Set "VNC Host Display" to `jet01.img.univie.ac.at:[DISPLAY]` +* Ser "Proxy/Gateway" to `[USERNAME]@jet01.img.univie.ac.at` +* Select "Use SSH". +* Connect. You'll be asked first for the SSH password, then for the VNC password. + +Option 2: [Bitvise SSH Client](https://www.bitvise.com/ssh-client-download) and RealVNC + +* Start the Bitvise SSH client +* Go to tab "C2S" +* Set "Listen Interface" to `127.0.0.1` +* Set "Listening Port" to `5900+[DISPLAY]`, e.g., `5905` +* Set "Destination Host" to `jet01.img.univie.ac.at` +* Set "Destination Port" to `5900+[DISPLAY]` +* Now start VncViewer and connect to `127.0.0.1:5900+[DISPLAY]` diff --git a/SSH-VPN-VNC/VNC.md b/SSH-VPN-VNC/VNC.md index 55261a5bfc53f8fc31d8213d46f2980be8639abe..6e1e708f5daeed8c59853f1e6f12ad1ed9bdc372 100644 --- a/SSH-VPN-VNC/VNC.md +++ b/SSH-VPN-VNC/VNC.md @@ -12,6 +12,7 @@ Xvnc is the Unix VNC server. Applications can display themselves on Xvnc as if i First of all check if a VNC server is already running or not. Depending on the results you have two options: 1. Use an existing. (Note the Port/Display Number) 2. Stop all and start a new VNC server +3. **on Jet** setup a service running your VNC Server. ```bash # Check VNC Server directly, lists available with DISPLAY/PORT @@ -27,6 +28,15 @@ vncserver -kill :[DISPLAY] vncserver ``` +#### Jet Cluser +on Jet there are the user services available to you: +```bash +# Help information on VNC userservice +userservices vnc -h +``` +if you have never used this script or a `vncserver` just running `userservices vnc` will be enough for you to setup the VNC service. It will tell you which port you have been automatically assigned. + + ### Connecting, setting the window manager Use a VNC client (e.g. the [RealVNC VNC Viewer](https://www.realvnc.com/en/connect/download/viewer/windows/) on Windows, or [Remmina](https://remmina.org/) on any Linux distribution) and connect to `srvx?.img.univie.ac.at:[DISPLAY]`. @@ -61,17 +71,37 @@ xterm -geometry -sb -sl 500 -fn 9x15bold -title "$VNCDESKTOP Desktop" & icewm & ``` +### VNC as a Service +This is only here for reference, on SRVX2 and Jet use the `userservices vnc`. + +Setup, replace `[DISPLAY]` with an appropriate number: +```bash +mkdir -p ~/.config/systemd/user +cp /usr/lib/systemd/user/vncserver@.service ~/.config/systemd/user/ +systemctl --user daemon-reload +vncpasswd +systemctl --user enable vncserver@:[DISPLAY].service --now +loginctl enable-linger +``` + Finally stop/restart the VNC server. ```bash -$ systemctl --user restart vncserver@:[DISPLAY].service --now +# Restart +systemctl --user restart vncserver@:[DISPLAY].service --now +# Stop +systemctl --user stop vncserver@:[DISPLAY].service --now +# Disable Service +systemctl --user disable vncserver@:[DISPLAY].service --now +# Enable Service +systemctl --user enable vncserver@:[DISPLAY].service --now ``` Monitor the status: ```bash # Like this -$ systemctl --user status vncserver@:[DISPLAY].service +systemctl --user status vncserver@:[DISPLAY].service # or -$ systemctl --user status vncserver.slice +systemctl --user status vncserver.slice vncserver@:7.service - Remote desktop service (VNC) Loaded: loaded (/jetfs/home/mblaschek/.config/systemd/user/vncserver@.service; enabled; vendor preset: e> @@ -91,7 +121,7 @@ $ systemctl --user status vncserver.slice Change the resolution to e.g. 1920x1080 (HD): ```bash -$ xrandr -s 1920x1080 -d $DISPLAY +xrandr -s 1920x1080 -d $DISPLAY ``` Adding resolutions according to your display's resolution have a look here: [add_xrandr_resolution.sh](add_xrandr_resolution.sh) diff --git a/VSC.md b/VSC.md index ff3c770657c4f7539e7be7bc852286280f18ac48..f7dc6765451b4f061ffb7c2b623d440571a32e7e 100644 --- a/VSC.md +++ b/VSC.md @@ -246,7 +246,9 @@ The VSC use the same software system as Jet and have environmental modules avail [username@l34 ~]$ module load <xyz> # load a particular package <xyz> into your session ``` - will load the intel compiler suite and add variables to your environment. **Please do not forget to add the module load statements to your jobs.** +on how to use environment modules go to [Using Environment Modules](Misc/Environment-Modules.md) + +