Download CMIP6 data from Google Cloud
28 Oct 2024This is just a post to locate resources I need. 🤔
As I am participating in the MDTF project, I need some sample climate data to test my POD. NOAA has a Google Cloud repository that stores CMIP6 data. To download the data, I need the gsutil installed on my linux machine.
I created the minimally necessary conda evironment by
conda create -n gcloud_cli python=3.11
I found a slide deck about NOAA’s public datasets online about how to read the data using Xarray:
datasets = []
for file in lets_get:
data_path = 'gs://' + bucket_name + '/' + file
ds3 = xr.open_dataset(fs.open(data_path), engine='h5netcdf')
datasets.append(ds3['TSkin'])
I downloaded two datasets
$ gsutil -m cp -r "gs://cmip6/CMIP6/CMIP/CAMS/CAMS-CSM1-0/historical/r1i1p1f2/Amon/ta" "gs://cmip6/CMIP6/CMIP/CAMS/CAMS-CSM1-0/historical/r1i1p1f2/Amon/ua" "gs://cmip6/CMIP6/CMIP/CAMS/CAMS-CSM1-0/historical/r1i1p1f2/Amon/va" .
$ gsutil -m cp -r "gs://cmip6/CMIP6/C4MIP/E3SM-Project/E3SM-1-1/hist-bgc/r1i1p1f1/Amon/ta" "gs://cmip6/CMIP6/C4MIP/E3SM-Project/E3SM-1-1/hist-bgc/r1i1p1f1/Amon/ua" "gs://cmip6/CMIP6/C4MIP/E3SM-Project/E3SM-1-1/hist-bgc/r1i1p1f1/Amon/va" .
Will give a try and report my findings here.
Additional information: