Skip to content
Snippets Groups Projects
Commit f05cf0de authored by Michael Blaschek's avatar Michael Blaschek :bicyclist:
Browse files

added odb, bufr

parent 34b6235c
No related branches found
No related tags found
No related merge requests found
Expand all Hide whitespace changes
Inline Side-by-side
Python/QA-010-Reading-Bufr-Synop.ipynb 0 → 100644
+ 824
− 0
View file @ f05cf0de
%% Cell type:markdown id:f4885378-6bf9-40b2-9046-9f1efd1e32ec tags:
# Reading BUFR files
Information about BUFR at [ECMWF](https://confluence.ecmwf.int/display/UDOC/BUFR+structure+-+ecCodes+BUFR+FAQ) inside ECCODES
%% Cell type:markdown id:4579e58c-b4cf-43f4-bd31-873598a81a95 tags:
install via
`pip install --user pdbufr`
or install the newest version
`pip install --user git+https://github.com/ecmwf/pdbufr.git`
it has eccodes as dependency
%% Cell type:code id:ecf6bff7-91e7-4cfa-8d7a-509e4e4ae229 tags:
``` python
import pdbufr
```
%% Cell type:code id:4935a745-b963-4aef-a8a1-dad695f3aa71 tags:
``` python
pdbufr.__version__
```
%% Output
'0.11.0'
%% Cell type:code id:e60003da-fc6b-4a9f-a568-53bc2beed9d0 tags:
``` python
SYNOP_BUFR_FILE = '/data/boden/BUFR/2022/01/31/ISMD12_LOWM.311200'
```
%% Cell type:code id:d3032d01-bbe4-49ca-bc43-117f3defa4ab tags:
``` python
keys = ['blockNumber',
'stationNumber',
'latitude',
'longitude',
'airTemperature',
'heightOfStationGroundAboveMeanSeaLevel',
'dewpointTemperature',
'windSpeed',
'windDirection',
'WMO_station_id',
'cloudType',
]
df_all = pdbufr.read_bufr(SYNOP_BUFR_FILE, columns=keys)
```
%% Cell type:code id:27f35abe-56f7-4afd-80b5-a3f93cc87b61 tags:
``` python
display(df_all)
```
%% Output
%% Cell type:markdown id:26cd0da1-9c5f-44c6-8527-09815102c014 tags:
## Reading Radiosonde information
%% Cell type:code id:df6a645b-01d2-499d-b2fc-0fe9cd0c393c tags:
``` python
df = pdbufr.read_bufr('/data/raso/BUFR/2022/01/31/IUSD04_LOWM.310300',
columns=('stationNumber', 'data_datetime', 'pressure', 'airTemperature', 'dewpointTemperature','windDirection', 'windSpeed',
'nonCoordinateGeopotentialHeight', 'timePeriod', 'longitude', 'latitude', 'height'),
)
```
%% Cell type:code id:f98cb422-02e2-4351-91d4-b40bc208266f tags:
``` python
display(df)
```
%% Output
%% Cell type:code id:2bbdbbb8-8aa6-495a-90ac-65f434119d8a tags:
``` python
```
Python/QA-011-Reading-ODB.ipynb 0 → 100644
+ 815
− 0
View file @ f05cf0de
This diff is collapsed.
Python/README.md
+ 10
− 0
View file @ f05cf0de
......@@ -135,6 +135,16 @@ You just need to add a configuration option and then you can open the Dashboard
If you need to get a better understanding of you functions memory and execution time, try these profiling options.
## Q: How to read BUFR files with python?
[BUFR](QA-010-Reading-Bufr-Synop.ipynb)
ECMWF created a python package that helps to read BUFR messages via ECCODES into pandas DataFrame.
## Q: How to read ODB files with python?
[ODB](QA-011-Reading-ODB.ipynb)
ECMWF created a python package that helps to read ODB messages via ODC or pure Python into pandas DataFrame.
## Q: How to ignore user site packages?
......
Python/bufr2synop.py deleted 100644 → 0
+ 0
− 15
View file @ 34b6235c
import eccodes
import pdbufr
keys = ['blockNumber',
'stationNumber',
'latitude',
'longitude',
'airTemperature',
'dewpointTemperature',
'windSpeed',
'windDirection',
]
df = pdbufr.read_bufr('ISMD22_LOWM.010000', columns=keys)
\ No newline at end of file
SSH-VPN-VNC/SSH.md
+ 1
− 1
View file @ f05cf0de
......@@ -77,7 +77,7 @@ Host a?-* a??-* hpc-* hpc2020-* ecs-*
and replacing `[USERNAME]` and `[u:account USERNAME]` with your usernames. Using such a file allows to connect like this `ssh srvx1` using the correct server adress and specified username. Copy this file as well on `login.univie.ac.at` and you can use commands like this: `ssh -t login ssh jet` to connect directly to `jet` via the `login` gateway.
Please note the special algorithms for ecaccess and of course ECMWF uses [teleport](../ECMWF.md#connecting-to-ecmwf-services) now.
Please note the special algorithms for ecaccess and of course ECMWF uses [teleport](../ECMWF/README.md#connecting-to-ecmwf-services) now.
**From eduroam**: You should be able to log in as above.
......
Servers/SRVX1.md
+ 2
− 2
View file @ f05cf0de
......@@ -10,7 +10,7 @@ Steps:
1. Request access / will be done for you by your supervisor.
2. As Staff, access using SSH - [How to SSH / VNC / VPN](../SSH-VPN-VNC/README.md)
3. As Student, access using Teaching Hub - [How to connect using the TeachingHub](../TeachingHub.md)
3. As Student, access using Teaching Hub - [How to connect using the TeachingHub](../Students/TeachingHub.md)
## System information
| Name | Value |
......@@ -39,7 +39,7 @@ Steps:
## Jupyterhub
<img src="../mkdocs/img/jupyterhub-logo.svg" width="150px">
SRVX1 serves a teaching [jupyterhub](https://jupyterhub.readthedocs.io/en/stable/) with a [jupyterlab](https://jupyterlab.readthedocs.io/en/stable/). It allows easy access for students and teachers. Access: [https://srvx1.img.univie.ac.at/hub](https://srvx1.img.univie.ac.at/hub)
SRVX1 serves a teaching [jupyterhub](https://jupyterhub.readthedocs.io/en/stable/) with a [jupyterlab](https://jupyterlab.readthedocs.io/en/stable/). It allows easy access for students and teachers. Access: [https://teachinghub.wolke.img.univie.ac.at](https://teachinghub.wolke.img.univie.ac.at)
Signup is only granted by teachers and requires a srvx1 user account. A new password is needed and a TOTP (time base one-time password) will be created.
......
Students/README.md
+ 19
− 3
View file @ f05cf0de
......@@ -5,20 +5,36 @@ To make you life easier we have listed a few things here that you should know to
???+ warning "since winter semester 2023"
There is the [old teachinghub](./TeachingHub.md) (will retire 31.3.2024) and the new teachinghub, which is handled via Moodle. Your lecturerer will give you access.
## Guidelines
Access to computational resources at IMGW is granted to the user by employment at the university or by a IMGW sponsor. Access is disabled on the termination date without further notice. An extension of the allocation may be negotiated with the responsible Admin or IMGW sponsor. The user accepts the following responsibilities:
- Computers and information systems must be used in an ethical and legal manner.
- The user agrees not to duplicate or use copyrighted or proprietary software without proper authorization.
- The user may not use computers and information systems in any manner for any business, professional, or other activity that is unrelated to the purpose of the resource allocation.
- The user is required to acknowledge the use of IMGW resources in any resulting publications.
- The user is responsible for protecting her/his access credentials and/or passwords.
- The user may not share her/his account privileges with anyone or knowingly permit any unauthorized access to a computer, computer privileges, systems, networks, or programs. The accounts of those involved will be disabled if sharing is detected.
- The user is responsible for backing up critical data to protect it against loss or corruption. The user is also responsible for understanding the usage and data retention policies for the file system and data archive resources used.
- The user agrees to report potential security breaches as soon as possible to the responsible Admin or IMGW sponsor.
- The user is responsible for ensuring that IMGW has her/his current contact information, including phone number, email address, and mailing address. If the user’s name, phone number, email address, or other information changes, the responsible Admin or IMGW sponsor must be promptly notified.
## Master Students
Great idea to start a master @ IMGW. Your supervisor will ask for an account for you and you will retrieve login credentials.
Great idea to start a master @ IMGW. Your supervisor will request a server account and you will retrieve login credentials. Please take a look at the [guidelines](#guidelines) above and acknowledge them.
There are three options:
As an employee or master student you can get access to these resources:
- access to [SRVX1](../Servers/SRVX1.md)
- access to [JET](../Servers/JET.md)
- access to [VSC](../VSC.md)
- acccess to [ECMWF](../ECMWF/README.md)
Most people at the department use [gitlab](https://gitlab.phaidra.org) and [mattermost](https://discuss.phaidra.org), which your supervisor can arange for you to get an account.
It might be useful to familiarize yourself with [git](../Git/README.md)
It might be useful to familiarize yourself with [git](../Git/README.md) and create a project dedicated to your master thesis on gitlab. Your supervisor can help you. Writing your thesis can be fascilitated by using [Overleaf](https://www.overleaf.de) and a [template](https://www.overleaf.com/read/ptpskhdqmqpt#566d55). As student at the University of Vienna, you can login with your u:account.
More template from our department can be found [here (wiki)](https://wiki.univie.ac.at/x/VBURC)
## Useful links
......
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Please register or to comment