Verification files

Bris-inference can generate verification files compatible with the Verif program (https://github.com/WFRT/verif). These files contain both forecasts and observations for point locations and together with the Verif tools let's you do extensive analysis of the performance of the model.

This can be configured by adding a Verif output in the config file:

outputs:
  - verif:
      filename: output_filename.nc
      variable: 2t
      obs_sources:
        - verif:
            filename: observation_file.nc

A Verif file can be produced for a single variable, which is defined by variable. This must be the name of a variable that the model produces. This can also be used for supported derived variables (currently only "ws" for wind-speed).

filename is the name of the output file. obs_sources is a list of observation sources, described next.

Observations

Verif NetCDF files typically contain forecasts and observations. In many cases, the user has available a Verif file containing observations. These files can be used to provide observations for this output.

However, Bris inference can also parse NetCDF files with a simpler format, containing only the observations. So if you do not already have Verif files we recommend that you dump observations and station metadata onto a NetCDF file with this structure:

   netcdf format {
   dimensions:
      time = UNLIMITED;
      location = 10;
   variables:
      int time(time);               // Valid time of forecast initialization in
                                    // number of seconds since 1970-01-01 00:00:00 +00:00
      float lat(location);          // Decimal degrees latitude
      float lon(location);          // Decimal degrees longitude
      float altitude(location);     // Altitude in meters
      float obs(time, location);    // Observations
      }

Verification against analysis

Bris inference can also be used to generate Verif files that compare forecast to the analysis (validation/test data). This is done by specifying an Anemoi zarr dataset as source to the verif output:

verif_dataset:
  dataset:
    dataset: PATH_TO_DATASET
    trim_edge: 160
  thinning: 20

outputs:
  - verif:
      filename: output_filename.nc
      variable: 2t
      obs_sources:
        - anemoidataset:
            dataset: ${verif_dataset}            
            variable: 2t

In the above example we sub-sample the verif dataset using the anemoi-datasets functions thinning and trim_edge. Bris inference will interpolate the prediction to the points in verif_dataset (or compare directly if the coordinates are in the prediction output).

Maximum interpolation distance

This output will interpolate the forecast fields to each observation point. In some cases, these observation points can be way outside the forecast field (for example if global observations are used to verify regional forecasts). This is not ideal for verifying the forecasts. This can be alleviated by setting max_distance to a distance in meters, which will throw out observation points that are too far away from the nearest forecast grid point.