diff --git a/docs/source/tutorial1.ipynb b/docs/source/tutorial1.ipynb index 798d6b941dc14d3bca583e5c14f92de2b8c18fdb..f425edbfe6521bad6b6ae9f8e88d3a51f8c15d65 100644 --- a/docs/source/tutorial1.ipynb +++ b/docs/source/tutorial1.ipynb @@ -2,144 +2,28 @@ "cells": [ { "cell_type": "markdown", - "id": "fd5c3005-f237-4495-9185-2d4d474cafd5", - "metadata": {}, - "source": [ - "# Tutorial 1: The assimilation step\n", - "DART-WRF is a python package which automates many things like configuration, saving configuration and output, handling computing resources, etc.\n", - "\n", - "The data for this experiment is accessible for students on the server srvx1.\n" - ] - }, - { - "cell_type": "markdown", - "id": "93d59d4d-c514-414e-81fa-4ff390290811", - "metadata": {}, + "id": "09371891-3ca3-404e-aeb3-7b58b900f563", + "metadata": { + "tags": [] + }, "source": [ - "### Configuring the experiment\n", - "Firstly, you need to configure the experiment in `config/cfg.py`.\n", - "\n", - "Let's go through the most important settings:\n", - "\n", - "- expname should be a unique identifier and will be used as folder name\n", - "- model_dx is the model resolution in meters\n", - "- n_ens is the ensemble size\n", - "- update_vars are the WRF variables which shall be updated by the assimilation\n", - "- filter_kind is 1 for the EAKF (see the DART documentation for more)\n", - "- prior and post_inflation defines what inflation we want (see the DART docs)\n", - "- sec is the statistical sampling error correction from Anderson (2012)\n", - "\n", - "```python\n", - "exp = utils.Experiment()\n", - "exp.expname = \"test_newcode\"\n", - "exp.model_dx = 2000\n", - "exp.n_ens = 10\n", - "exp.update_vars = ['U', 'V', 'W', 'THM', 'PH', 'MU', 'QVAPOR', 'QCLOUD', 'QICE', 'PSFC']\n", - "exp.filter_kind = 1\n", - "exp.prior_inflation = 0\n", - "exp.post_inflation = 4\n", - "exp.sec = True\n", - "\n", - "```\n", - "In case you want to generate new observations like for an observing system simulations experiment, OSSE), set \n", - "```python\n", - "exp.use_existing_obsseq = False`.\n", - "```\n", - "\n", - "`exp.nature` defines which WRF files will be used to draw observations from, e.g.: \n", - "```python\n", - "exp.nature = '/users/students/lehre/advDA_s2023/data/sample_nature/'\n", - "```\n", - "\n", - "`exp.input_profile` is used, if you create initial conditions from a so called wrf_profile (see WRF guide).\n", - "```python\n", - "exp.input_profile = '/doesnt_exist/initial_profiles/wrf/ens/raso.fc.<iens>.wrfprof'\n", - "```\n", - "\n", - "\n", - "For horizontal localization half-width of 20 km and 3 km vertically, set\n", - "```python\n", - "exp.cov_loc_vert_km_horiz_km = (3, 20)\n", - "```\n", - "You can also set it to False for no vertical localization.\n", - "\n", - "#### Single observation\n", - "Set your desired observations like this. \n", - "```python\n", - "t = dict(plotname='Temperature', plotunits='[K]',\n", - " kind='RADIOSONDE_TEMPERATURE', \n", - " n_obs=1, # number of observations\n", - " obs_locations=[(45., 0.)], # location of observations\n", - " error_generate=0.2, # observation error used to generate observations\n", - " error_assimilate=0.2, # observation error used for assimilation\n", - " heights=[1000,], # for radiosondes, use range(1000, 17001, 2000)\n", - " cov_loc_radius_km=50) # horizontal localization half-width\n", - "\n", - "exp.observations = [t,] # select observations for assimilation\n", - "```\n", - "\n", - "#### Multiple observations\n", - "To generate a grid of observations, use\n", - "```python\n", - "vis = dict(plotname='VIS 0.6µm', plotunits='[1]',\n", - " kind='MSG_4_SEVIRI_BDRF', sat_channel=1, \n", - " n_obs=961, obs_locations='square_array_evenly_on_grid',\n", - " error_generate=0.03, error_assimilate=0.03,\n", - " cov_loc_radius_km=20)\n", - "```\n", - "\n", - "But caution, n_obs should only be one of the following:\n", - "\n", - "- 22500 for 2km observation density/resolution \n", - "- 5776 for 4km; \n", - "- 961 for 10km; \n", - "- 256 for 20km; \n", - "- 121 for 30km\n", - "\n", - "For vertically resolved data, like radar, n_obs is the number of observations at each observation height level." + "# Test1" ] }, { - "cell_type": "markdown", - "id": "16bd3521-f98f-4c4f-8019-31029fd678ae", + "cell_type": "code", + "execution_count": 1, + "id": "ab8f6d86-dc41-4d5c-b6d9-a9f396c1ec70", "metadata": {}, + "outputs": [], "source": [ - "### Configuring the hardware\n", - "In case you use a cluster which is not supported, configure paths inside `config/clusters.py`.\n", - "\n", - "\n", - "\n", - "\n", - "### Assimilate observations\n", - "We start by importing some modules:\n", - "```python\n", - "import datetime as dt\n", - "from dartwrf.workflows import WorkFlows\n", - "```\n", - "\n", - "To assimilate observations at dt.datetime `time` we set the directory paths and times of the prior ensemble forecasts:\n", - "\n", - "```python\n", - "prior_path_exp = '/users/students/lehre/advDA_s2023/data/sample_ensemble/'\n", - "prior_init_time = dt.datetime(2008,7,30,12)\n", - "prior_valid_time = dt.datetime(2008,7,30,12,30)\n", - "assim_time = prior_valid_time\n", - "```\n", - "\n", - "Finally, we run the data assimilation by calling\n", - "```python\n", - "w = WorkFlows(exp_config='cfg.py', server_config='srvx1.py')\n", - "\n", - "w.assimilate(assim_time, prior_init_time, prior_valid_time, prior_path_exp)\n", - "```\n", - "\n", - "Congratulations! You're done!" + "import os, sys" ] }, { "cell_type": "code", "execution_count": null, - "id": "82e809a8-5972-47f3-ad78-6290afe4ae17", + "id": "3bc1d633-cf91-46ad-96d2-256dda6fa335", "metadata": {}, "outputs": [], "source": [] diff --git a/docs/source/tutorial2.ipynb b/docs/source/tutorial2.ipynb index df58b78b35dd5099e742d2e50933d4395150e716..74576f45257c7f7b0f6fc111bbd41461dcce1c29 100644 --- a/docs/source/tutorial2.ipynb +++ b/docs/source/tutorial2.ipynb @@ -2,100 +2,23 @@ "cells": [ { "cell_type": "markdown", - "id": "fd5c3005-f237-4495-9185-2d4d474cafd5", + "id": "02ac2a1f-272e-4f23-960d-e7f95d683b53", "metadata": { "tags": [] }, "source": [ - "# Tutorial 2: Forecast after DA\n", - "\n", - "\n", - "**Goal**: To run a cycled data assimilation experiment.\n", - "[`cycled_exp.py`](https://github.com/lkugler/DART-WRF/blob/master/generate_free.py) contains an example which will be explained here:\n", - "\n", - "Now there are two options:\n", - "1) To start a forecast from an existing forecast, i.e. from WRF restart files\n", - "2) To start a forecast from defined thermodynamic profiles, i.e. from a `wrf_profile`\n", - "\n", - "\n", - "### Restart a forecast\n", - "To run a forecast from initial conditions of a previous forecasts, we import these modules\n", - "```python\n", - "import datetime as dt\n", - "from dartwrf.workflows import WorkFlows\n", - "```\n", - "\n", - "Let's say you want to run a forecast starting at 9 UTC until 12 UTC.\n", - "Initial conditions shall be taken from a previous experiment in `/user/test/data/sim_archive/exp_abc` which was initialized at 6 UTC and there are WRF restart files for 9 UTC.\n", - "Then the code would be\n", - "\n", - "```python\n", - "prior_path_exp = '/user/test/data/sim_archive/exp_abc'\n", - "prior_init_time = dt.datetime(2008,7,30,6)\n", - "prior_valid_time = dt.datetime(2008,7,30,9)\n", - "\n", - "w = WorkFlows(exp_config='cfg.py', server_config='srvx1.py')\n", - "\n", - "begin = dt.datetime(2008, 7, 30, 9)\n", - "end = dt.datetime(2008, 7, 30, 12)\n", - "\n", - "w.prepare_WRFrundir(begin)\n", - "\n", - "w.prepare_IC_from_prior(prior_path_exp, prior_init_time, prior_valid_time)\n", - "\n", - "w.run_ENS(begin=begin, # start integration from here\n", - " end=end, # integrate until here\n", - " output_restart_interval=9999, # do not write WRF restart files\n", - " )\n", - "```\n", - "Note that we use predefined workflow functions like `run_ENS`.\n", - "\n", - "\n", - "### Forecast run after Data Assimilation\n", - "In order to continue after assimilation you need the posterior = prior (1) + increments (2)\n", - "\n", - "1. Set posterior = prior\n", - "```python\n", - "id = w.prepare_IC_from_prior(prior_path_exp, prior_init_time, prior_valid_time, depends_on=id)\n", - "```\n", - "\n", - "2) Update posterior with increments from assimilation\n", - "After this, the wrfrst files are updated with assimilation increments from DART output and copied to the WRF's run directories so you can continue to run the forecast ensemble.\n", - "```python\n", - "id = w.update_IC_from_DA(time, depends_on=id)\n", - "```\n", - "\n", - "3) Define how long you want to run the forecast and when you want WRF restart files. Since they take a lot of space, we want as few as possible.\n", - "\n", - "```python\n", - "timedelta_integrate = dt.timedelta(hours=5)\n", - "output_restart_interval = 9999 # any value larger than the forecast duration\n", - "```\n", - "\n", - "If you run DA in cycles of 15 minutes, it will be\n", - "```python\n", - "timedelta_integrate = dt.timedelta(hours=5)\n", - "timedelta_btw_assim = dt.timedelta(minutes=15)\n", - "output_restart_interval = timedelta_btw_assim.total_seconds()/60\n", - "```\n", - "\n", - "\n", - "3) Run WRF ensemble\n", - "```python\n", - "id = w.run_ENS(begin=time, # start integration from here\n", - " end=time + timedelta_integrate, # integrate until here\n", - " output_restart_interval=output_restart_interval,\n", - " depends_on=id)\n", - "```\n" + "# Test2" ] }, { "cell_type": "code", - "execution_count": null, - "id": "400244f1-098b-46ea-b29d-2226c7cbc827", + "execution_count": 1, + "id": "afdee309-2ee0-418d-89db-47d057386a29", "metadata": {}, "outputs": [], - "source": [] + "source": [ + "import os, sys" + ] } ], "metadata": { diff --git a/docs/source/tutorial3.ipynb b/docs/source/tutorial3.ipynb index d3018ab95b378770ca3b05b1e2fc9adb547a188e..f88f409d13ec50b58df3fdff437a2c8751107f78 100644 --- a/docs/source/tutorial3.ipynb +++ b/docs/source/tutorial3.ipynb @@ -2,107 +2,23 @@ "cells": [ { "cell_type": "markdown", - "id": "fd5c3005-f237-4495-9185-2d4d474cafd5", + "id": "8ced2b8b-6829-4b16-acb7-03fd1b8b0ff8", "metadata": { "tags": [] }, "source": [ - "# Tutorial 3: Cycle forecast and assimilation\n", - "\n", - "\n", - "**Goal**: To run a cycled data assimilation experiment.\n", - "[`cycled_exp.py`](https://github.com/lkugler/DART-WRF/blob/master/generate_free.py) contains an example which will be explained here:\n", - "\n", - "For example, your experiment can look like this\n", - "\n", - "```python\n", - "prior_path_exp = '/jetfs/home/lkugler/data/sim_archive/exp_v1.19_P2_noDA'\n", - "\n", - "init_time = dt.datetime(2008, 7, 30, 13)\n", - "time = dt.datetime(2008, 7, 30, 14)\n", - "last_assim_time = dt.datetime(2008, 7, 30, 14)\n", - "forecast_until = dt.datetime(2008, 7, 30, 14, 15)\n", - "\n", - "w.prepare_WRFrundir(init_time)\n", - "id = w.run_ideal(depends_on=id)\n", - "\n", - "prior_init_time = init_time\n", - "prior_valid_time = time\n", - "\n", - "while time <= last_assim_time:\n", - "\n", - " # usually we take the prior from the current time\n", - " # but one could use a prior from a different time from another run\n", - " # i.e. 13z as a prior to assimilate 12z observations\n", - " prior_valid_time = time\n", - "\n", - " id = w.assimilate(time, prior_init_time, prior_valid_time, prior_path_exp, depends_on=id)\n", - "\n", - " # 1) Set posterior = prior\n", - " id = w.prepare_IC_from_prior(prior_path_exp, prior_init_time, prior_valid_time, depends_on=id)\n", - "\n", - " # 2) Update posterior += updates from assimilation\n", - " id = w.update_IC_from_DA(time, depends_on=id)\n", - "\n", - " # How long shall we integrate?\n", - " timedelta_integrate = timedelta_btw_assim\n", - " output_restart_interval = timedelta_btw_assim.total_seconds()/60\n", - " if time == last_assim_time: #this_forecast_init.minute in [0,]: # longer forecast every full hour\n", - " timedelta_integrate = forecast_until - last_assim_time # dt.timedelta(hours=4)\n", - " output_restart_interval = 9999 # no restart file after last assim\n", - "\n", - " # 3) Run WRF ensemble\n", - " id = w.run_ENS(begin=time, # start integration from here\n", - " end=time + timedelta_integrate, # integrate until here\n", - " output_restart_interval=output_restart_interval,\n", - " depends_on=id)\n", - "\n", - " # as we have WRF output, we can use own exp path as prior\n", - " prior_path_exp = cluster.archivedir \n", - "\n", - " id_sat = w.create_satimages(time, depends_on=id)\n", - "\n", - " # increment time\n", - " time += timedelta_btw_assim\n", - "\n", - " # update time variables\n", - " prior_init_time = time - timedelta_btw_assim\n", - " \n", - "w.verify_sat(id_sat)\n", - "w.verify_wrf(id)\n", - "w.verify_fast(id)\n", - "```\n", - "\n", - "#### Job scheduling status\n", - "If you work on a server with a queueing system, the script submits jobs into the SLURM queue with dependencies so that SLURM starts the jobs itself as soon as resources are available. Most jobs need only a few cores, but model integration is done across many nodes. You can look at the status with\n", - "```bash\n", - "$ squeue -u `whoami` --sort=i\n", - " JOBID PARTITION NAME USER ST TIME NODES NODELIST(REASON)\n", - " 1710274 mem_0384 prepwrfr lkugler PD 0:00 1 (Priority)\n", - " 1710275 mem_0384 IC-prior lkugler PD 0:00 1 (Dependency)\n", - " 1710276 mem_0384 Assim-42 lkugler PD 0:00 1 (Dependency)\n", - " 1710277 mem_0384 IC-prior lkugler PD 0:00 1 (Dependency)\n", - " 1710278 mem_0384 IC-updat lkugler PD 0:00 1 (Dependency)\n", - " 1710279 mem_0384 preWRF2- lkugler PD 0:00 1 (Dependency)\n", - " 1710280_[1-10] mem_0384 runWRF2- lkugler PD 0:00 1 (Dependency)\n", - " 1710281 mem_0384 pRTTOV-6 lkugler PD 0:00 1 (Dependency)\n", - " 1710282 mem_0384 Assim-3a lkugler PD 0:00 1 (Dependency)\n", - " 1710283 mem_0384 IC-prior lkugler PD 0:00 1 (Dependency)\n", - " 1710284 mem_0384 IC-updat lkugler PD 0:00 1 (Dependency)\n", - " 1710285 mem_0384 preWRF2- lkugler PD 0:00 1 (Dependency)\n", - " 1710286_[1-10] mem_0384 runWRF2- lkugler PD 0:00 1 (Dependency)\n", - " 1710287 mem_0384 pRTTOV-7 lkugler PD 0:00 1 (Dependency)\n", - "```\n", - "\n" + "# Test3" ] }, { "cell_type": "code", - "execution_count": null, - "id": "400244f1-098b-46ea-b29d-2226c7cbc827", + "execution_count": 1, + "id": "b8d43e28-c9cb-403e-915c-79266b5a03c7", "metadata": {}, "outputs": [], - "source": [] + "source": [ + "import os, sys" + ] } ], "metadata": {