If you need access, talk to your supervisor to create an account for you. You will get a username and a password as well as an OTP device (hardware or smartphone). Accounts are handled via [www.ecmwf.int](https://www.ecmwf.int/user/login)
Available Services @ IMGW:
...
...
@@ -15,8 +14,7 @@ Available Services @ IMGW:
## Connecting to ECMWF Services
???+ warning "Teleport supported versions"
Please note that ECMWF does currently only support teleport version up to 13. [ecmwf information](https://confluence.ecmwf.int/display/UDOC/Teleport+SSH+Access)
Please note that ECMWF does currently only support teleport version up to 13. [ecmwf information](https://confluence.ecmwf.int/display/UDOC/Teleport+SSH+Access)
### from home
...
...
@@ -38,7 +36,7 @@ If browser window does not open automatically, open it by clicking on the link:
> Profile URL: https://jump.ecmwf.int:443
Logged in as: [MAIL ADDRESS]
Cluster: jump.ecmwf.int
Roles:
Roles:
Logins: [ECMWF USERNAME]
Kubernetes: disabled
Valid until: 2024-10-09 23:18:33 +0200 CEST [valid for 11h58m0s]
...
...
@@ -47,7 +45,6 @@ If browser window does not open automatically, open it by clicking on the link:
look at the [SSH config](#configuration) below and you should be fine to connect.
### from IMGW
A ECMWF user can connect to the ECS/ATOS using teleport, first load the teleport module and start the ssh-agent:
It is required to have an SSH-agent running in order to connect to the ECMWF servers. The teleport module includes a `startagent` function to allow to reconnect to an existing ssh-agent. Do not start too many agents!
```bash title="start ssh-agent"
...
...
@@ -130,7 +126,6 @@ userservices sshtools -h
userservices sshtools -k
```
## ECMWF Access Server (ECS)
There is an issue with ssh-keys
...
...
@@ -151,6 +146,23 @@ Sometimes there are also different issues with the connection. You can [search t
- Example - [Teleport permission denied](https://gitlab.phaidra.org/imgw/computer-resources/-/issues/61#note_36373)
- Example - [Teleport authentication failed](https://gitlab.phaidra.org/imgw/computer-resources/-/issues/58)
## Storage at ECMWF
There is the information for the new ATOS system: [ecmwf wiki](https://confluence.ecmwf.int/display/UDOC/HPC2020%3A+Filesystems).
A short summart
| File System | Suitable for ... | Technology | Features | Quota |
| HOME | permanent files, e.g. profile, utilities, sources, libraries, etc. | NFS | It is backed up. | **10GB** |
| PERM | permanent files. no backup. | NFS | *DO NOT USE IN PARALLEL APPLICATIONS. DO NOT USE FOR JOB STANDARD OUTPUT/ERROR* | **500 GB** |
| HPCPERM | permanent files. no backup | Lustre | | 100 GB for users without HPC access. **1 TB** for users with HPC access |
| SCRATCH | all temporary (large) files. Main storage for your jobs and experiments input and output files. | Lustre | Automatic deletion after 30 days of last access. no backup. | **50 TB** for users with HPC access. 2 TB for users without HPC access |
| SCRATCHDIR | Big temporary data for an individual session or job, not as fast as TMPDIR but higher capacity. Files accessible from all cluster. | Lustre | Deleted at the end of session or jobCreated per session/ job as a subdirectory in SCRATCH | part of SCRATCH quota |
| TMPDIR | Fast temporary data for an individual session or job, small files only. Local to every node. | SSD | Deleted at the end of session or job. Created per session/ job | space and limits are shared with LOCALSSD |
Typically all users from Austria a grouped together into the `at` group and can share data with each other.
## Connecting via ECaccess
...
...
@@ -212,7 +224,7 @@ if you have troubles or for some other reason, if you remove this file `~/.eccer
```sh title="ECAccess gateway"
# check what server you are connected to
$ ecaccess-gateway-name
$ ecaccess-gateway-name
boaccess.ecmwf.int
# connected ?
$ ecaccess-gateway-connected
...
...
@@ -246,16 +258,16 @@ $ ecaccess-ectrans-list
$ ecaccess-ectrans-list
176674485 COPY aurora boaccess.ecmwf.int Oct 09 09:42
```
if you encounter a STOP or ERROR, then you can also check the gateway ([boaccess](https://boaccess.ecmwf.int), [imgaccess](https://ecaccess.wolke.img.univie.ac.at)) to have a look at the message (file transfers).
if you encounter a STOP or ERROR, then you can also check the gateway ([boaccess](https://boaccess.ecmwf.int), [imgaccess](https://ecaccess.wolke.img.univie.ac.at)) to have a look at the message (file transfers).
This file serves as a template only the first part is important.
Change:
-`active='yes'`
-`comment='Ssomething that explains where is will send the data to'`
-`directory='/srvfs/scratch/[USERNAME]'` or another directory.
-`hostName='login.img.univie.ac.at'` or `jet01...` or `jet02...`
-`login='[USERNAME]'`
-`protocol='genericSftp'`
save the file and then you can add this to the correct gateway. **Remember that JET is only available from the gateway (ecaccess.img.univie.ac.at)**, which is available only from inside the VPN@UNIVIE under [ecaccess.wolke.img.univie.ac.at](https://ecaccess.wolke.img.univie.ac.at).
-`active='yes'`
-`comment='Ssomething that explains where is will send the data to'`
-`directory='/srvfs/scratch/[USERNAME]'` or another directory.
-`hostName='login.img.univie.ac.at'` or `jet01...` or `jet02...`
-`login='[USERNAME]'`
-`protocol='genericSftp'`
save the file and then you can add this to the correct gateway. **Remember that JET is only available from the gateway (ecaccess.img.univie.ac.at)**, which is available only from inside the VPN@UNIVIE under [ecaccess.wolke.img.univie.ac.at](https://ecaccess.wolke.img.univie.ac.at).
@@ -342,8 +356,6 @@ jet jet01.img.univie.ac.at active scratch
```
## ECaccess Gateway
The department is running a member state ecaccess gateway service. **The purpose of an individual access server is to bridge ECMWF's network with IMGW's network.** Hence, protecting these networks. For example, you can access the JET cluster from the department ecaccess server, but not from boaccess server, but from boaccess you can accesss aurora.
