{
  "cells": [
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "collapsed": false
      },
      "outputs": [],
      "source": [
        "%matplotlib inline"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "\n# Blended forecast\n\nThis tutorial shows how to construct a blended forecast from an ensemble nowcast\nusing the STEPS approach and a Numerical Weather Prediction (NWP) rainfall\nforecast. The used datasets are from the Bureau of Meteorology, Australia.\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "collapsed": false
      },
      "outputs": [],
      "source": [
        "import os\nfrom datetime import datetime\n\nimport numpy as np\nfrom matplotlib import pyplot as plt\n\nimport pysteps\nfrom pysteps import io, rcparams, blending\nfrom pysteps.visualization import plot_precip_field"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "## Read the radar images and the NWP forecast\n\nFirst, we import a sequence of 3 images of 10-minute radar composites\nand the corresponding NWP rainfall forecast that was available at that time.\n\nYou need the pysteps-data archive downloaded and the pystepsrc file\nconfigured with the data_source paths pointing to data folders.\nAdditionally, the pysteps-nwp-importers plugin needs to be installed, see\nhttps://github.com/pySTEPS/pysteps-nwp-importers.\n\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "collapsed": false
      },
      "outputs": [],
      "source": [
        "# Selected case\ndate_radar = datetime.strptime(\"202010310400\", \"%Y%m%d%H%M\")\n# The last NWP forecast was issued at 00:00\ndate_nwp = datetime.strptime(\"202010310000\", \"%Y%m%d%H%M\")\nradar_data_source = rcparams.data_sources[\"bom\"]\nnwp_data_source = rcparams.data_sources[\"bom_nwp\"]"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "### Load the data from the archive\n\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "collapsed": false
      },
      "outputs": [],
      "source": [
        "root_path = radar_data_source[\"root_path\"]\npath_fmt = \"prcp-c10/66/%Y/%m/%d\"\nfn_pattern = \"66_%Y%m%d_%H%M00.prcp-c10\"\nfn_ext = radar_data_source[\"fn_ext\"]\nimporter_name = radar_data_source[\"importer\"]\nimporter_kwargs = radar_data_source[\"importer_kwargs\"]\ntimestep = 10.0\n\n# Find the radar files in the archive\nfns = io.find_by_date(\n    date_radar, root_path, path_fmt, fn_pattern, fn_ext, timestep, num_prev_files=2\n)\n\n# Read the radar composites\nimporter = io.get_method(importer_name, \"importer\")\nradar_precip, _, radar_metadata = io.read_timeseries(fns, importer, **importer_kwargs)\n\n# Import the NWP data\nfilename = os.path.join(\n    nwp_data_source[\"root_path\"],\n    datetime.strftime(date_nwp, nwp_data_source[\"path_fmt\"]),\n    datetime.strftime(date_nwp, nwp_data_source[\"fn_pattern\"])\n    + \".\"\n    + nwp_data_source[\"fn_ext\"],\n)\n\nnwp_importer = io.get_method(\"bom_nwp\", \"importer\")\nnwp_precip, _, nwp_metadata = nwp_importer(filename)\n\n# Only keep the NWP forecasts from the last radar observation time (2020-10-31 04:00)\n# onwards\n\nnwp_precip = nwp_precip[24:43, :, :]"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "## Pre-processing steps\n\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "collapsed": false
      },
      "outputs": [],
      "source": [
        "# Make sure the units are in mm/h\nconverter = pysteps.utils.get_method(\"mm/h\")\nradar_precip, radar_metadata = converter(radar_precip, radar_metadata)\nnwp_precip, nwp_metadata = converter(nwp_precip, nwp_metadata)\n\n# Threshold the data\nradar_precip[radar_precip < 0.1] = 0.0\nnwp_precip[nwp_precip < 0.1] = 0.0\n\n# Plot the radar rainfall field and the first time step of the NWP forecast.\ndate_str = datetime.strftime(date_radar, \"%Y-%m-%d %H:%M\")\nplt.figure(figsize=(10, 5))\nplt.subplot(121)\nplot_precip_field(\n    radar_precip[-1, :, :],\n    geodata=radar_metadata,\n    title=f\"Radar observation at {date_str}\",\n)\nplt.subplot(122)\nplot_precip_field(\n    nwp_precip[0, :, :], geodata=nwp_metadata, title=f\"NWP forecast at {date_str}\"\n)\nplt.tight_layout()\nplt.show()\n\n# transform the data to dB\ntransformer = pysteps.utils.get_method(\"dB\")\nradar_precip, radar_metadata = transformer(radar_precip, radar_metadata, threshold=0.1)\nnwp_precip, nwp_metadata = transformer(nwp_precip, nwp_metadata, threshold=0.1)\n\n# r_nwp has to be four dimentional (n_models, time, y, x).\n# If we only use one model:\nif nwp_precip.ndim == 3:\n    nwp_precip = nwp_precip[None, :]"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "For the initial time step (t=0), the NWP rainfall forecast is not that different\nfrom the observed radar rainfall, but it misses some of the locations and\nshapes of the observed rainfall fields. Therefore, the NWP rainfall forecast will\ninitially get a low weight in the blending process.\n\n### Determine the velocity fields\n\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "collapsed": false
      },
      "outputs": [],
      "source": [
        "oflow_method = pysteps.motion.get_method(\"lucaskanade\")\n\n# First for the radar images\nvelocity_radar = oflow_method(radar_precip)\n\n# Then for the NWP forecast\nvelocity_nwp = []\n# Loop through the models\nfor n_model in range(nwp_precip.shape[0]):\n    # Loop through the timesteps. We need two images to construct a motion\n    # field, so we can start from timestep 1. Timestep 0 will be the same\n    # as timestep 1.\n    _v_nwp_ = []\n    for t in range(1, nwp_precip.shape[1]):\n        v_nwp_ = oflow_method(nwp_precip[n_model, t - 1 : t + 1, :])\n        _v_nwp_.append(v_nwp_)\n        v_nwp_ = None\n    # Add the velocity field at time step 1 to time step 0.\n    _v_nwp_ = np.insert(_v_nwp_, 0, _v_nwp_[0], axis=0)\n    velocity_nwp.append(_v_nwp_)\nvelocity_nwp = np.stack(velocity_nwp)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "## The blended forecast\n\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "collapsed": false
      },
      "outputs": [],
      "source": [
        "precip_forecast = blending.steps.forecast(\n    precip=radar_precip,\n    precip_models=nwp_precip,\n    velocity=velocity_radar,\n    velocity_models=velocity_nwp,\n    timesteps=18,\n    timestep=timestep,\n    issuetime=date_radar,\n    n_ens_members=1,\n    precip_thr=radar_metadata[\"threshold\"],\n    kmperpixel=radar_metadata[\"xpixelsize\"] / 1000.0,\n    noise_stddev_adj=\"auto\",\n    vel_pert_method=None,\n)\n\n# Transform the data back into mm/h\nprecip_forecast, _ = converter(precip_forecast, radar_metadata)\nradar_precip, _ = converter(radar_precip, radar_metadata)\nnwp_precip, _ = converter(nwp_precip, nwp_metadata)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "### Visualize the output\n\nThe NWP rainfall forecast has a lower weight than the radar-based extrapolation\nforecast at the issue time of the forecast (+0 min). Therefore, the first time\nsteps consist mostly of the extrapolation.\nHowever, near the end of the forecast (+180 min), the NWP share in the blended\nforecast has become more important and the forecast starts to resemble the\nNWP forecast more.\n\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "collapsed": false
      },
      "outputs": [],
      "source": [
        "fig = plt.figure(figsize=(5, 12))\n\nleadtimes_min = [30, 60, 90, 120, 150, 180]\nn_leadtimes = len(leadtimes_min)\nfor n, leadtime in enumerate(leadtimes_min):\n\n    # Nowcast with blending into NWP\n    plt.subplot(n_leadtimes, 2, n * 2 + 1)\n    plot_precip_field(\n        precip_forecast[0, int(leadtime / timestep) - 1, :, :],\n        geodata=radar_metadata,\n        title=f\"Nowcast +{leadtime} min\",\n        axis=\"off\",\n        colorbar=False,\n    )\n\n    # Raw NWP forecast\n    plt.subplot(n_leadtimes, 2, n * 2 + 2)\n    plot_precip_field(\n        nwp_precip[0, int(leadtime / timestep) - 1, :, :],\n        geodata=nwp_metadata,\n        title=f\"NWP +{leadtime} min\",\n        axis=\"off\",\n        colorbar=False,\n    )"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "### References\n\nBowler, N. E., and C. E. Pierce, and A. W. Seed. 2004. \"STEPS: A probabilistic\nprecipitation forecasting scheme which merges an extrapolation nowcast with\ndownscaled NWP.\" Forecasting Research Technical Report No. 433. Wallingford, UK.\n\nBowler, N. E., and C. E. Pierce, and A. W. Seed. 2006. \"STEPS: A probabilistic\nprecipitation forecasting scheme which merges an extrapolation nowcast with\ndownscaled NWP.\" Quarterly Journal of the Royal Meteorological Society 132(16):\n2127-2155. https://doi.org/10.1256/qj.04.100\n\nSeed, A. W., and C. E. Pierce, and K. Norman. 2013. \"Formulation and evaluation\nof a scale decomposition-based stochastic precipitation nowcast scheme.\" Water\nResources Research 49(10): 6624-664. https://doi.org/10.1002/wrcr.20536\n\n"
      ]
    }
  ],
  "metadata": {
    "kernelspec": {
      "display_name": "Python 3",
      "language": "python",
      "name": "python3"
    },
    "language_info": {
      "codemirror_mode": {
        "name": "ipython",
        "version": 3
      },
      "file_extension": ".py",
      "mimetype": "text/x-python",
      "name": "python",
      "nbconvert_exporter": "python",
      "pygments_lexer": "ipython3",
      "version": "3.8.6"
    }
  },
  "nbformat": 4,
  "nbformat_minor": 0
}