OSOP API documentation¶

This is where the code for OSOP modules is documented.

compare_terciles¶

This module contains routines for tercile calculations

Functions to compare terciles for forecasts and hindcasts.

osop.compare_terciles.compute_forecast(config, downloaddir, products_hindcast, products_forecast)¶

Calculate tercile forecast data for 1 month forecasts.

Parameters:

config (dict) – A dictionary containing the configuration parameters.
downloaddir (str) – The path to the download directory of the forecasts grib.
products_hindcast (str) – The path to the tercile categories from compute_products.
products_forecast (str) – The output location for the products generated.

Return type:

None

osop.compare_terciles.mask_cat(fcst, terciles)¶

Create a boolean mask for where the forecast value falls.

Parameters:

fcst (xarray.DataArray) – The forecast array.
terciles (xarray.Dataset) – The hindcast terciles.

Returns:

The boolean masks for (lower, higher, middle) categories.

Return type:

tuple of xarray.DataArray

osop.compare_terciles.mme_process_forecasts(months, suffix, Services, services_values, productsfcdir, config, services_weights)¶

Load each tercile forecast and combine for MME.

Parameters:

months (int or None) – Set to None or value of month based on leads.
suffix (str) – Used for naming between nmonths or the imonth lead.
Services (dict) – List of services to combine.
services_values (dict) – List of service values i.e. ecmwf 51.
productsfcdir (str) – The location for the files to output to and get from.
config (dict) – The configuration parameters for the forecast.
services_weights (dict) – The weight of the service in context of mme.

Returns:

The combined MME forecast array.

Return type:

xarray.DataArray

osop.compare_terciles.mme_products(Services, config, productsfcdir)¶

Load each tercile forecast and combine for MME.

Parameters:

Services (dict) – List of services to combine.
config (dict) – The configuration parameters for the forecast.
productsfcdir (str) – The location for the files to output to and get from.

Return type:

None

Notes

Saves array (xarray.DataArray) - The multi-model ensemble forecast percentages.

osop.compare_terciles.nth_month(forecast_data, hindcast_terciles, products_forecast, forecast_fname, config)¶

Produce a nth month tercile forecast.

This takes data in the form of an xarray that contains the month, the percentage and the lat-lon coordinates.

Parameters:

forecast_data (xarray.Dataset) – The re-indexed forecast data.
hindcast_terciles (xarray.Dataset) – The x-array that contains the matching tercile categories.
products_forecast (str) – The location for the files to output to.
forecast_fname (str) – The name of the forecast data.

Return type:

None

Notes

Saves output data-array that contains the percent values for each tercile and co-ord.

osop.compare_terciles.one_month(forecast_data, hindcast_terciles, products_forecast, forecast_fname)¶

Produce a one month tercile forecast.

This takes data in the form of an xarray that contains the month, the percentage and the lat-lon coordinates.

Parameters:

forecast_data (xarray.Dataset) – The re-indexed forecast data.
hindcast_terciles (xarray.Dataset) – The x-array that contains the matching tercile categories.
products_forecast (str) – The location for the files to output to.
forecast_fname (str) – The name of the forecast data.

Return type:

None

Notes

Saves output data-array that contains the percent values for each tercile and co-ord.

osop.compare_terciles.percentage(array)¶

Take a boolean mask for a forecast dataset and return the percentage of Trues.

Parameters:: array (xarray.DataArray) – The input boolean mask.
Returns:: The percentage values.
Return type:: xarray.DataArray

osop.compare_terciles.update_config(origin, systemfc, config)¶

Create a copy of the config dict to be used for repeated load in of tercile forecasts.

Parameters:

origin (str) – The service to be loaded.
systemfc (str) – The service version.
config (dict) – The dictionary to be copied and variated.

Returns:

The copy of the dictionary with new updated names.

Return type:

dict

compute_products_func¶

Functions to make forecasts and hindcasts products.

osop.compute_products_func.calc_anoms(hcst, hcst_bname, config, productsdir)¶

Calculate anomalies and save them to netCDF files.

Parameters:

hcst (xarray.Dataset) – Hindcast dataset.
hcst_bname (str) – Base name of the hindcast grib file.
config (dict) – Configuration parameters.
productsdir (str) – Directory path to save the netCDF files.

Returns:

The original hindcast data and the n-month aggregated data.

Return type:

tuple of xarray.Dataset

Notes

Saves 1 month and n month anomalies to netCDF files.

osop.compute_products_func.calc_products(config, downloaddir, productsdir)¶

Calculate anomalies and tercile probabilities for a given hindcast dataset.

Parameters:

config (dict) – Configuration parameters.
downloaddir (str) – Directory path to save the netCDF files.
productsdir (str) – Directory path to save the netCDF files.

Return type:

None

osop.compute_products_func.calc_products_mme(services, config, productsdir)¶

Calculate anomalies and tercile probabilities for the MME combined data.

Parameters:

services (dict) – All the services intended for the MME.
config (dict) – Configuration parameters.
productsdir (str) – Directory path to save the netCDF files.

Return type:

None

osop.compute_products_func.get_thresh(icat, quantiles, xrds, dims=['number', 'start_date'])¶

Calculate the boundaries of forecast categories defined by quantiles (e.g. terciles).

Parameters:

icat (int) – The category number. 0 (lower than), 1 (normal), 2 (higher than).
quantiles (list of float) – The list of quantiles. Use [1/3., 2/3.] for terciles.
xrds (xarray.Dataset) – The dataset containing the hindcast data.
dims (list of str, optional) – The dimensions to consider when calculating the quantiles. Defaults to [‘number’, ‘start_date’].

Returns:

A tuple containing the lower and upper boundaries for the forecast category.

Return type:

tuple of xarray.DataArray or float

Raises:

ValueError – If any of the specified dimensions are not present in the dataset.

osop.compute_products_func.mme_products_hindcast(services, config, productsdir)¶

Load each indexed dataset and combine for MME.

Parameters:

services (dict) – List of services to combine.
config (dict) – The configuration parameters for the forecast.
productsdir (str) – The location for the files to output to and get from.

Return type:

None

Notes

Saves array (xarray.DataArray) - The multi-model ensemble forecast percentages.

osop.compute_products_func.prob_terc(config, hcst_bname, hcst, hcst_nm, productsdir)¶

Calculate probabilities for tercile categories.

Counts members within each category and saves them to netCDF files. This function computes the tercile thresholds for both 1-month and n-month aggregated hindcast data and saves them to netCDF files.

Parameters:

config (dict) – Configuration parameters.
hcst_bname (str) – Basename of hindcast file.
hcst (xarray.Dataset) – The dataset containing the hindcast data.
hcst_nm (xarray.Dataset) – The dataset containing the n-month aggregated hindcast data.
productsdir (str) – Directory path to save the netCDF files.

Return type:

None

Notes

Saves tercile forecasts to netCDF file.

osop.compute_products_func.process_mme_products(array, output, aggr, config, sig, member_weight=0.1)¶

Calculate anomalies and save them to netCDF files.

Parameters:

array (xarray.DataArray) – Target array.
output (dict) – Enters as an empty array.
aggr (str) – ‘1m’ or ‘nm’ array.
config (dict) – Configuration parameters.
sig (str) – Signifier for addition to save path for mean, anom or tercile mme.
member_weight (float, optional) – The fractional weight for the service, i.e. weight/sum of weights.

Returns:

xarray.DataArray – The combined mme for 1 month and n month combined.
str – The save name for the output file.

osop.compute_products_func.valid_time(output, save_name)¶

Input Valid_time into the array for later.

Parameters:

output (xarray.DataArray) – Target array.
save_name (str) – Save path.

Returns:

The updated array.

Return type:

xarray.DataArray

constants¶

Collection of constants used in the OSOP library.

Notes

Defines named domains as dictionaries of longitude and latitude bounds.

ens_plotting¶

Collection of plotting codes relevant to ensembles.

osop.ens_plotting.fc_title(config, multim=False)¶

Create a title string for the forecast plot based on the configuration.

Parameters:

config (dict) – Dictionary containing configuration parameters.
multim (bool) – True or False for grouped forecasts.

Returns:

Formatted title string.

Return type:

str

osop.ens_plotting.get_cmap(precip_cs=False, wmo_cs=True)¶

Return 3 colormaps for below, normal and above tercile forecasts.

Parameters:

precip_cs (bool, optional) – If True, return a colorscale for rainfall, else for temperature, by default False.
wmo_cs (bool, optional) – If True, use WMO colorscales, else use custom, by default True.

Returns:

Tuple of three colormaps: (below, normal, above).

Return type:

tuple

osop.ens_plotting.plot_forecasts(productdir, plotsdir, config)¶

Call functions and parse configurations for plotting forecasts.

Parameters:

productdir (str) – Location for dataset to be plotted.
plotsdir (str) – Location for plots to save to.
config (dict) – Dictionary for parameters of the file/dataset.

Return type:

None

osop.ens_plotting.plot_tercile_fc(mme, atitle, centre, mme_svc, var='precipitation', mask=None, map_setting='False')¶

Plot a tercile forecast.

Uses different colormaps for each of three terciles. Uses a threshold of 40% below which it does not plot.

Parameters:

mme (xarray.Dataset) – Tercile forecast dataset.
atitle (str) – Title for the plot.
var (str, optional) – Variable name in the dataset, by default “precipitation”.
mask (xarray.DataArray, optional) – Optional dry mask as a DataArray, by default None.
map_setting (object or str, optional) – Optional map feature from cartopy, by default “False”.

Returns:

The matplotlib figure object.

Return type:

matplotlib.figure.Figure

osop.ens_plotting.reformatt(data, variable)¶

Reformatt a forecast_percentage dataset to be able to run through ens_plotting routines.

Parameters:

data (xarray.Dataset) – The forecast percentage dataset.
variable (str) – The variable name to use in the reformatted dataset.

Returns:

A reformatted version of the forecast data for ens_plotting functions.

Return type:

xarray.Dataset

osop.ens_plotting.truncate_colormap(cmap, minval=0.0, maxval=1.0, n=100)¶

Given a colormap, truncate it at bottom (minval) or top (maxval).

Parameters:

cmap (matplotlib.colors.Colormap) – Matplotlib color map to be truncated.
minval (float, optional) – Minimum value to truncate at (0 to 1), by default 0.0.
maxval (float, optional) – Maximum value to truncate at (0 to 1), by default 1.0.
n (int, optional) – Number of color levels, by default 100.

Returns:

New truncated colormap.

Return type:

matplotlib.colors.LinearSegmentedColormap

plot_verify¶

Plotting functions for verification plots.

osop.plot_verify.corr_plots(scoresdir, plotdir, hcst_bname, aggr, config, score, titles, method)¶

Plot deterministic scores.

Parameters:

scoresdir (str) – The directory to fetch the input files from.
plotdir (str) – The directory to save the plot.
hcst_bname (str) – The basename of the hindcast file.
aggr (str) – The aggregation period.
config (dict) – Configuration parameters.
score (str) – The name of the score.
titles (list) – Titles for the plot.
method (str) – Plotting method.

Return type:

None

osop.plot_verify.generate_plots(config, titles, scoresdir, plotdir, method)¶

Generate verification plots for the given configuration.

Parameters:

config (dict) – Configuration parameters.
titles (list) – Titles for the plot.
scoresdir (str) – Directory containing score files.
plotdir (str) – Directory to save plots.
method (str) – Plotting method.

Return type:

None

osop.plot_verify.location(config)¶

Prepare location specific POV borders for plots based on arguments in the config dictionary.

Parameters:: config (dict) – Dictionary containing the configuration parameters.
Returns:: A Natural Earth data set name to go into the axis plot that’s downloaded based on location. If no location set (i.e. None), no borders will plot. If the name is misspelt then a KeyError will be raised suggesting a check of location entry in the shell script.
Return type:: object or str

Notes

Natural Earth has a download issue that searches for a file that doesn’t exist; a partial import is managed regardless. On second run - as this file is not used and download has already happened - the plot will work fine. To avoid a second run each time a new data set is imported the try/except does the import for no reason and then the finally is used after to generate the plot. - This is a Natural Earth Specific problem that can be removed when fixed. Relevant to Cartopy issue #2319 , #2477 and #2534 - when resolved can be removed.

osop.plot_verify.plot_rel(score_f, score_fname, config, score, plotdir, titles, score_title)¶

Plot reliability diagram.

Parameters:

score_f (numpy.ndarray) – The reliability score data.
score_fname (str) – The score filename.
config (dict) – Configuration parameters.
score (str) – The name of the score.
plotdir (str) – The directory to save the plot.
titles (list) – Titles for the plot.
score_title (str) – The name for the plot in the file directory.

Return type:

None

osop.plot_verify.plot_score(score_f, score_fname, category, config, score, titles, plotdir, score_title)¶

Plot the score on a map.

Parameters:

score_f (numpy.ndarray) – The score data.
score_fname (str) – The score filename.
category (str or int or None) – The tercile category. Set to None if not relevant for the score.
config (dict) – Configuration parameters.
score (str) – The name of the score.
titles (list) – Titles for the plot.
plotdir (str) – The directory to save the plot.
score_title (str) – The name for the plot in the file directory.

Return type:

None

osop.plot_verify.prep_titles(config, scoresdir)¶

Prepare titles for the plot based on the arguments in the config dictionary.

Currently this replicates the titles here: https://confluence.ecmwf.int/display/CKB/C3S+seasonal+forecasts+verification+plots

Parameters:: config (dict) – Dictionary containing the configuration parameters.
Returns:: Tuple containing the prepared titles for the plot: (first line, second line, third line).
Return type:: tuple of str

regridders¶

Functions to help with regridding for xarray.

osop.regridders.interp_target(domain, res)¶

Create an interpolation target for a specific domain.

Parameters:

domain (dict) – Dictionary containing x0, x1, y0, y1.
res (float) – Resolution of the target grid in degrees.

Returns:

Dataset to use as interpolation target.

Return type:

xarray.Dataset

Raises:

ValueError – If x0 >= x1 or y0 >= y1.

osop.regridders.regrid_data_std(input_ds, target_ds)¶

Regrid dataset to match target grid resolution.

Regrids the dataset appropriately for its type (planned expansion for precipitation).

Parameters:

input_ds (xarray.Dataset) – Data to be re-gridded.
target_ds (xarray.Dataset) – Data set with the target grid.

Returns:

output_ds (xarray.Dataset) – Regridded dataset to be used for analysis.
target_ds (xarray.Dataset) – Matching target dataset (no changes).

Raises:

KeyError – If alignment fails due to incompatible datasets.

util¶

A module with utility functions for seasonal forecasts.

osop.util.get_tindex(infile)¶

Use eccodes to check if there is an indexing time dimension.

Parameters:: infile (str) – Name of file to check.
Returns:: Name of time dimension to use for indexing. Returns ‘time’ for burst ensemble and ‘indexing_time’ for lagged.
Return type:: str

osop.util.index(forecast_local, st_dim_name)¶

Reindex and restyle the forecast grib.

This ensures that the data layout is consistent and compatible with hindcast terciles.

Parameters:

forecast_local (str) – File location for the grib file.
st_dim_name (str) – Name of the start date dimension (important for lagged models).

Returns:

A re-indexed x-array for forecast data.

Return type:

xarray.Dataset

osop.util.season_stats(dataset, start_year, end_year, stats=['mean'])¶

Calculate the climatology mean and quantiles for a dataset.

This is done over a given time period for standard meteorological seasons and the categories of the terciles for each season.

Parameters:

dataset (xarray.Dataset) – The data to calculate the stats on.
start_year (int) – The start year of the period to calculate the stats.
end_year (int) – The end year of the period to calculate the stats.
stats (list, optional) – The statistics to calculate, default is [‘mean’]. Options: [‘mean’, ‘seas_ts’], where seas_ts is the seasonal time series for all years.

Returns:

A dictionary with the calculated stats as xarray.Datasets.

Return type:

dict

osop.util.sel_season_time(dataset, start_year, end_year)¶

Extract data for specified years from Dec of first year to Nov of last year.

Parameters:

dataset (xarray.Dataset) – The data to extract the time from.
start_year (int) – The start year of the period to extract.
end_year (int) – The end year of the period to extract.

Returns:

The data with the time extracted.

Return type:

xarray.Dataset

OSOP API documentation¶

compare_terciles¶

compute_products_func¶

constants¶

ens_plotting¶

plot_verify¶

regridders¶

util¶

OSOP

Navigation

Related Topics