Skip to content
GitLab
Explore
Sign in
Primary navigation
Search or go to…
Project
TRACMIP
Manage
Activity
Members
Labels
Plan
Issues
Issue boards
Milestones
Wiki
Code
Merge requests
Repository
Branches
Commits
Tags
Repository graph
Compare revisions
Snippets
Deploy
Releases
Package registry
Model registry
Operate
Terraform modules
Monitor
Incidents
Service Desk
Analyze
Value stream analytics
Contributor analytics
Repository analytics
Model experiments
Help
Help
Support
GitLab documentation
Compare GitLab plans
Community forum
Contribute to GitLab
Provide feedback
Terms and privacy
Keyboard shortcuts
?
Snippets
Groups
Projects
Show more breadcrumbs
Climate
TRACMIP
Commits
df21827e
Commit
df21827e
authored
4 years ago
by
Aiko Voigt
Browse files
Options
Downloads
Patches
Plain Diff
Not working first example of loading data from Pangeo cloud
parent
617474fd
No related branches found
Branches containing commit
No related tags found
No related merge requests found
Changes
1
Hide whitespace changes
Inline
Side-by-side
Showing
1 changed file
pangeo/load_data.ipynb
+252
-0
252 additions, 0 deletions
pangeo/load_data.ipynb
with
252 additions
and
0 deletions
pangeo/load_data.ipynb
0 → 100644
+
252
−
0
View file @
df21827e
{
"cells": [
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# How to access and load TRACMIP data from the Pangeo cloud\n",
"\n",
"1. \"load\" the Tracmip collection \n",
"2. get some basic info on Tracmip collection\n",
"3. load monthly mean precip for the aquaControl simulation\n",
"4. plot meridional zonal-mean time-mean profile for one model"
]
},
{
"cell_type": "code",
"execution_count": 1,
"metadata": {},
"outputs": [],
"source": [
"from matplotlib import pyplot as plt\n",
"import numpy as np\n",
"import pandas as pd\n",
"import xarray as xr\n",
"import zarr\n",
"import gcsfs\n",
"\n",
"xr.set_options(display_style='html')\n",
"%matplotlib inline\n",
"%config InlineBackend.figure_format = 'retina' "
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"df = pd.read_csv('https://storage.googleapis.com/cmip6/tracmip.csv')\n",
"df.head()"
]
},
{
"cell_type": "code",
"execution_count": 3,
"metadata": {},
"outputs": [
{
"ename": "NameError",
"evalue": "name 'df' is not defined",
"output_type": "error",
"traceback": [
"\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
"\u001b[0;31mNameError\u001b[0m Traceback (most recent call last)",
"\u001b[0;32m<ipython-input-3-36bdea2f3883>\u001b[0m in \u001b[0;36m<module>\u001b[0;34m\u001b[0m\n\u001b[0;32m----> 1\u001b[0;31m \u001b[0mdf_pr\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mdf\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mquery\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m\"frequency == 'Amon' & variable == 'pr' & experiment == 'aquaControl'\"\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m",
"\u001b[0;31mNameError\u001b[0m: name 'df' is not defined"
]
}
],
"source": [
"df_pr = df.query(\"frequency == 'Amon' & variable == 'pr' & experiment == 'aquaControl'\")"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## 1. \"Load\" Tracmip collection"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"from intake import open_catalog\n",
"\n",
"# get whole pangeo catalogue\n",
"cat = open_catalog(\"https://raw.githubusercontent.com/pangeo-data/pangeo-datastore/master/intake-catalogs/climate.yaml\")\n",
"# select tracmip collection\n",
"col = cat.tracmip()"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## 2. Basic info on the collection"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"print collection to screen: this shows that there is 3 output frequencies (monthly-mean, daily-mean, 3-hr snapshots), \n",
"11 experiments (6 are due to the CALTECH model with changed atmosperic opacity), and 47 variables"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"col"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"print starting and end portion of the collection"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"col.df.head()"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"col.df.tail()"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"print some further information on the collection (i.e., dataframe)"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"col.df.columns.unique()"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"col.df.model.unique()\n"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"col.df.experiment.unique()"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## 3. Now actually load the monthly-mean precip data for the aquaControl experiment, use a dictionary for this"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"note: the option \"zarr_kwargs={'consolidated': True}\" for to_dataset_dicts does not seem necessary but is still included here"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"ds_dict = col.search(frequency=\"Amon\", experiment=\"aquaControl\",\n",
" variable=\"pr\").to_dataset_dict(zarr_kwargs={'consolidated': True})"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## 4. Plot zonal-mean time-mean precip for last 20 years for CNRM-AM5 model"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"import matplotlib.pyplot as plt"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"ds_dict['CNRM-AM5.aquaControl.Amon']['pr']"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"plt.plot(ds_dict['CNRM-AM5.aquaControl.Amon'].lat, \n",
" ds_dict['CNRM-AM5.aquaControl.Amon']['pr'].isel(time=slice(120,360)).mean(['lon', 'time'])*86400)\n",
"plt.xlabel('degree latitude')\n",
"plt.ylabel('precipitation (mm/day)')\n",
"plt.title('CNRM-AM5.aquaControl.Amon')"
]
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 3",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.8.5"
}
},
"nbformat": 4,
"nbformat_minor": 4
}
%% Cell type:markdown id: tags:
# How to access and load TRACMIP data from the Pangeo cloud
1.
"load" the Tracmip collection
2.
get some basic info on Tracmip collection
3.
load monthly mean precip for the aquaControl simulation
4.
plot meridional zonal-mean time-mean profile for one model
%% Cell type:code id: tags:
```
python
from
matplotlib
import
pyplot
as
plt
import
numpy
as
np
import
pandas
as
pd
import
xarray
as
xr
import
zarr
import
gcsfs
xr
.
set_options
(
display_style
=
'
html
'
)
%
matplotlib
inline
%
config
InlineBackend
.
figure_format
=
'
retina
'
```
%% Cell type:code id: tags:
```
python
df
=
pd
.
read_csv
(
'
https://storage.googleapis.com/cmip6/tracmip.csv
'
)
df
.
head
()
```
%% Cell type:code id: tags:
```
python
df_pr
=
df
.
query
(
"
frequency ==
'
Amon
'
& variable ==
'
pr
'
& experiment ==
'
aquaControl
'"
)
```
%% Output
---------------------------------------------------------------------------
NameError Traceback (most recent call last)
<ipython-input-3-36bdea2f3883> in <module>
----> 1 df_pr = df.query("frequency == 'Amon' & variable == 'pr' & experiment == 'aquaControl'")
NameError: name 'df' is not defined
%% Cell type:markdown id: tags:
## 1. "Load" Tracmip collection
%% Cell type:code id: tags:
```
python
from
intake
import
open_catalog
# get whole pangeo catalogue
cat
=
open_catalog
(
"
https://raw.githubusercontent.com/pangeo-data/pangeo-datastore/master/intake-catalogs/climate.yaml
"
)
# select tracmip collection
col
=
cat
.
tracmip
()
```
%% Cell type:markdown id: tags:
## 2. Basic info on the collection
%% Cell type:markdown id: tags:
print collection to screen: this shows that there is 3 output frequencies (monthly-mean, daily-mean, 3-hr snapshots),
11 experiments (6 are due to the CALTECH model with changed atmosperic opacity), and 47 variables
%% Cell type:code id: tags:
```
python
col
```
%% Cell type:markdown id: tags:
print starting and end portion of the collection
%% Cell type:code id: tags:
```
python
col
.
df
.
head
()
```
%% Cell type:code id: tags:
```
python
col
.
df
.
tail
()
```
%% Cell type:markdown id: tags:
print some further information on the collection (i.e., dataframe)
%% Cell type:code id: tags:
```
python
col
.
df
.
columns
.
unique
()
```
%% Cell type:code id: tags:
```
python
col
.
df
.
model
.
unique
()
```
%% Cell type:code id: tags:
```
python
col
.
df
.
experiment
.
unique
()
```
%% Cell type:markdown id: tags:
## 3. Now actually load the monthly-mean precip data for the aquaControl experiment, use a dictionary for this
%% Cell type:markdown id: tags:
note: the option "zarr_kwargs={'consolidated': True}" for to_dataset_dicts does not seem necessary but is still included here
%% Cell type:code id: tags:
```
python
ds_dict
=
col
.
search
(
frequency
=
"
Amon
"
,
experiment
=
"
aquaControl
"
,
variable
=
"
pr
"
).
to_dataset_dict
(
zarr_kwargs
=
{
'
consolidated
'
:
True
})
```
%% Cell type:markdown id: tags:
## 4. Plot zonal-mean time-mean precip for last 20 years for CNRM-AM5 model
%% Cell type:code id: tags:
```
python
import
matplotlib.pyplot
as
plt
```
%% Cell type:code id: tags:
```
python
ds_dict
[
'
CNRM-AM5.aquaControl.Amon
'
][
'
pr
'
]
```
%% Cell type:code id: tags:
```
python
plt
.
plot
(
ds_dict
[
'
CNRM-AM5.aquaControl.Amon
'
].
lat
,
ds_dict
[
'
CNRM-AM5.aquaControl.Amon
'
][
'
pr
'
].
isel
(
time
=
slice
(
120
,
360
)).
mean
([
'
lon
'
,
'
time
'
])
*
86400
)
plt
.
xlabel
(
'
degree latitude
'
)
plt
.
ylabel
(
'
precipitation (mm/day)
'
)
plt
.
title
(
'
CNRM-AM5.aquaControl.Amon
'
)
```
This diff is collapsed.
Click to expand it.
Preview
0%
Loading
Try again
or
attach a new file
.
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Save comment
Cancel
Please
register
or
sign in
to comment